Relating Thermodynamics to Information Theory: The Equality of Free Energy and Mutual Information

Citation

Feinstein, David I. (1986) Relating Thermodynamics to Information Theory: The Equality of Free Energy and Mutual Information. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/XVQB-7902. https://resolver.caltech.edu/CaltechTHESIS:03212013-084718067

Abstract

In this thesis we uncover a new relation which links thermodynamics and information theory. We consider time as a channel and the detailed state of a physical system as a message. As the system evolves with time, ever present noise insures that the "message" is corrupted. Thermodynamic free energy measures the approach of the system toward equilibrium. Information theoretical mutual information measures the loss of memory of initial state. We regard the free energy and the mutual information as operators which map probability distributions over state space to real numbers. In the limit of long times, we show how the free energy operator and the mutual information operator asymptotically attain a very simple relationship to one another. This relationship is founded on the common appearance of entropy in the two operators and on an identity between internal energy and conditional entropy. The use of conditional entropy is what distinguishes our approach from previous efforts to relate thermodynamics and information theory.

Thesis Files