A characterization of generalized multinomial coefficients related to the entropic chain rule

Abstract

There is an asymptotic correspondence between the multiplicative relations among multinomial coefficients and the (additive) recursive property of Shannon entropy known as the chain rule. We show that both types of identities are manifestations of a unique algebraic construction: a 1-cocycle condition in information cohomology, an algebraic invariant of presheaves of modules on certain categories of observables. Depending on the coefficients, the 1-cocycles can be information measures (Shannon entropy, Tsallis α-entropy) or generalized (Fontené-Ward) multinomial coefficients. In each case the 1-cocycle condition encodes a system of functional equations. We obtain in particular a combinatorial analogue of the "fundamental equation of information theory": a simple functional equation that uniquely characterizes the generalized binomial coefficients. The asymptotic correspondence mentioned above extends to any α-entropy and certain multinomial coefficients with compatible asymptotic behavior, shedding new light on the meaning of the chain rule and its deformations.

Additional details