Thermodynamics and Information Theory - Essay Example

Running Head: Thermodynamics and Information Theory Thermodynamics and Information Theory of IntroductionThere is a remarkable similarity between the mathematical expressions for the thermodynamic entropy, usually denoted by S, of a physical system in the thermodynamics established by Ludwig Boltzmann in 1875; and the information theoretic entropy, which is normally expressed as H, of Ralph and Shannon a developed in the 1942. Shannon, In the beginning Shannon was not initially aware of this similarity, and later commented on it upon publicizing information theory in A Mathematical Theory of Communication.

In my paper I will explores what links there are between the two different concepts, and how far they can be regarded as connected in practical and theory.Theoretical RelationshipDespite the similarities between the two concepts there is a very important difference between the two. The information entropy which is denoted as H can be calculated for almost any probability distribution and if the message is taken to be that the event which is represented by n had probability pi takes place, out of the space of the list of events which are possible.

But the thermodynamic entropy S refers to thermodynamic probabilities pi in particular. (Skyttner, 2005) In addition to this the thermodynamic entropy is dominated by different kinds, space and arrangements of the system, and particularly its energy, which are possible and especially on a molecular scale. When we compare we see that, information entropy of any macroscopic event is so small as to be completely useless and irrelevant. (Skyttner, 2005) We can also make a connection between the two and if the probabilities related to any question are the thermodynamic probabilities.

Gibbs entropy represented by can then be seen as simply the amount of Shannon information needed to analyze and define the microscopic state of the system in great details, or we can say its been given its macroscopic description. Or, as the Lewis writing about chemical entropy in 1936, the Gain in entropy will always mean loss of information and nothing more than that. In order to be more concrete, we use discrete case using base of two different logarithms, the reduced Gibbs entropy is equal to the minimum number of yes or no questions that must be answered in order to completely specify the microstate of a system, provided we know the microstate of a particular system In addition to this the prescription to find the equilibrium distributions of statistical mechanics, for example, the Boltzmann distribution, simply by maximizing the Gibbs entropy state to exact constraints which are called the Gibbs algorithm, these days its not seen as something not unique to laws of thermodynamics, but as a principle of general relevance in all sorts of statistical inference.

( Skyttner, 2005) Conclusion In theory there is a remarkable relationship between the laws of thermodynamics and the information systems. Both have their significance and they depend on each other in many different ways. References:Skyttner, Lars. (2005). General systems theory. World Scientific Pub Co Inc.