Entropy of Highly Correlated Quantized Data
- Creators
- Marco, Daniel
- Neuhoff, David L.
Abstract
This paper considers the entropy of highly correlated quantized samples. Two results are shown. The first concerns sampling and identically scalar quantizing a stationary continuous-time random process over a finite interval. It is shown that if the process crosses a quantization threshold with positive probability, then the joint entropy of the quantized samples tends to infinity as the sampling rate goes to infinity. The second result provides an upper bound to the rate at which the joint entropy tends to infinity, in the case of an infinite-level uniform threshold scalar quantizer and a stationary Gaussian random process. Specifically, an asymptotic formula for the conditional entropy of one quantized sample conditioned on the previous quantized sample is derived. At high sampling rates, these results indicate a sharp contrast between the large encoding rate (in bits/sec) required by a lossy source code consisting of a fixed scalar quantizer and an ideal, sampling-rate-adapted lossless code, and the bounded encoding rate required by an ideal lossy source code operating at the same distortion.
Additional Information
© 2010 IEEE. Manuscript received August 30, 2006; revised August 21, 2008. Current version published April 21, 2010. This work was supported in part by NSF Grants ANI-0112801 and CCF 0329715 and in part by the Center for the Mathematics of Information at California Institute of Technology. Portions of this work were presented at the IEEE International Symposium on Information Theory, Adelaide, Australia, July 2005.Attached Files
Published - Marco2010p10345Ieee_T_Inform_Theory.pdf
Files
Name | Size | Download all |
---|---|---|
md5:1d96dbd6242564acc8c8f35f74d62cfd
|
570.8 kB | Preview Download |
Additional details
- Eprint ID
- 18700
- Resolver ID
- CaltechAUTHORS:20100616-093118997
- ANI-0112801
- NSF
- CCF 0329715
- NSF
- Center for the Mathematics of Information at Caltech
- Created
-
2010-06-17Created from EPrint's datestamp field
- Updated
-
2021-11-08Created from EPrint's last_modified field
- Other Numbering System Name
- INSPEC Accession Number
- Other Numbering System Identifier
- 11256627