When is Shannon's lower bound tight at finite blocklength?

Abstract

This paper formulates an abstract version of Shannon's lower bound that applies to abstract sources and arbitrary distortion measures and that recovers the classical Shannon lower bound as a special case. A necessary and sufficient condition for it to be attained exactly is presented. It is demonstrated that whenever that condition is met, the d-tilted information of the source adopts a simple, explicit representation that parallels Shannon's lower bound. That convenient representation simplifies the non-asymptotic analysis of achievable rate-distortion tradeoffs. In particular, if a memoryless source meets Shannon's lower bound with equality, then its rate-dispersion function is given simply by the varentropy of the source.

Additional details