Rényi information dimension: Fundamental limits of almost lossless analog compression

Yihong Wu; Sergio Verdú

doi:10.1109/TIT.2010.2050803

Rényi information dimension: Fundamental limits of almost lossless analog compression

Yihong Wu, Sergio Verdú

Research output: Contribution to journal › Article › peer-review

131 Scopus citations

Abstract

In Shannon theory, lossless source coding deals with the optimal compression of discrete sources. Compressed sensing is a lossless coding strategy for analog sources by means of multiplication by real-valued matrices. In this paper we study almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor. The fundamental limit is shown to the information dimension proposed by Rnyi in 1959.

Original language	English (US)
Article number	5508633
Pages (from-to)	3721-3748
Number of pages	28
Journal	IEEE Transactions on Information Theory
Volume	56
Issue number	8
DOIs	https://doi.org/10.1109/TIT.2010.2050803
State	Published - Aug 2010

All Science Journal Classification (ASJC) codes

Information Systems
Computer Science Applications
Library and Information Sciences

Keywords

Analog compression
Rnyi information dimension
Shannon theory
compressed sensing
information measures
source coding

Access to Document

10.1109/TIT.2010.2050803

Cite this

@article{4b7685389d4d409aad40b3f185935e4e,

title = "R{\'e}nyi information dimension: Fundamental limits of almost lossless analog compression",

abstract = "In Shannon theory, lossless source coding deals with the optimal compression of discrete sources. Compressed sensing is a lossless coding strategy for analog sources by means of multiplication by real-valued matrices. In this paper we study almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor. The fundamental limit is shown to the information dimension proposed by Rnyi in 1959.",

keywords = "Analog compression, Rnyi information dimension, Shannon theory, compressed sensing, information measures, source coding",

author = "Yihong Wu and Sergio Verd{\'u}",

note = "Funding Information: Manuscript received March 02, 2009; revised April 30, 2010. Date of current version July 14, 2010. This work was supported in part by the National Science Foundation under Grants CCF-0635154 and CCF-0728445. The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Seoul, Korea, July 2009 [55].",

year = "2010",

month = aug,

doi = "10.1109/TIT.2010.2050803",

language = "English (US)",

volume = "56",

pages = "3721--3748",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "8",

}

TY - JOUR

T1 - Rényi information dimension

T2 - Fundamental limits of almost lossless analog compression

AU - Wu, Yihong

AU - Verdú, Sergio

N1 - Funding Information: Manuscript received March 02, 2009; revised April 30, 2010. Date of current version July 14, 2010. This work was supported in part by the National Science Foundation under Grants CCF-0635154 and CCF-0728445. The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Seoul, Korea, July 2009 [55].

PY - 2010/8

Y1 - 2010/8

N2 - In Shannon theory, lossless source coding deals with the optimal compression of discrete sources. Compressed sensing is a lossless coding strategy for analog sources by means of multiplication by real-valued matrices. In this paper we study almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor. The fundamental limit is shown to the information dimension proposed by Rnyi in 1959.

AB - In Shannon theory, lossless source coding deals with the optimal compression of discrete sources. Compressed sensing is a lossless coding strategy for analog sources by means of multiplication by real-valued matrices. In this paper we study almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor. The fundamental limit is shown to the information dimension proposed by Rnyi in 1959.

KW - Analog compression

KW - Rnyi information dimension

KW - Shannon theory

KW - compressed sensing

KW - information measures

KW - source coding

UR - http://www.scopus.com/inward/record.url?scp=77954596738&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954596738&partnerID=8YFLogxK

U2 - 10.1109/TIT.2010.2050803

DO - 10.1109/TIT.2010.2050803

M3 - Article

AN - SCOPUS:77954596738

SN - 0018-9448

VL - 56

SP - 3721

EP - 3748

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 8

M1 - 5508633

ER -

Rényi information dimension: Fundamental limits of almost lossless analog compression

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this