The Rate-Distortion Risk in Estimation from Compressed Data

Alon Kipnis, Stefano Rini, Andrea J. Goldsmith

Research output: Contribution to journalArticlepeer-review

Abstract

Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the RD risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions are simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data.

Original languageEnglish (US)
Article number9387338
Pages (from-to)2910-2924
Number of pages15
JournalIEEE Transactions on Information Theory
Volume67
Issue number5
DOIs
StatePublished - May 2021

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Keywords

  • Compression
  • Estimation
  • Rate distortion
  • Source coding
  • Transport theory

Fingerprint Dive into the research topics of 'The Rate-Distortion Risk in Estimation from Compressed Data'. Together they form a unique fingerprint.

Cite this