TY - GEN
T1 - Estimation-theoretic representation of mutual information
AU - Palomar, Daniel P.
AU - Verdú, Sergio
N1 - Funding Information:
This work was supported in part by the Fulbright Program and the Ministry of Education and Science of Spain; the U.S. National Science Foundation under Grant NCR-0074277; and through collaborative participation in the Communications and Networks Consortium sponsored by the U.S. Army Research Laboratory under the Collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0011. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon.
PY - 2005
Y1 - 2005
N2 - A fundamental relationship between information theory and estimation theory was recently unveiled for the Gaussian channel, relating the derivative of mutual information with the minimum mean-square error. This paper generalizes this fundamental link between information theory and estimation theory to arbitrary channels and in particular encompasses the discrete memoryless channel (DMC). In addition to the intrinsic theoretical interest of such a result, it naturally leads to an efficient numerical computation of mutual information for cases in which it was previously infeasible such as with LDPC codes.
AB - A fundamental relationship between information theory and estimation theory was recently unveiled for the Gaussian channel, relating the derivative of mutual information with the minimum mean-square error. This paper generalizes this fundamental link between information theory and estimation theory to arbitrary channels and in particular encompasses the discrete memoryless channel (DMC). In addition to the intrinsic theoretical interest of such a result, it naturally leads to an efficient numerical computation of mutual information for cases in which it was previously infeasible such as with LDPC codes.
UR - http://www.scopus.com/inward/record.url?scp=84961889288&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84961889288&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84961889288
T3 - 43rd Annual Allerton Conference on Communication, Control and Computing 2005
SP - 2013
EP - 2022
BT - 43rd Annual Allerton Conference on Communication, Control and Computing 2005
PB - University of Illinois at Urbana-Champaign, Coordinated Science Laboratory and Department of Computer and Electrical Engineering
T2 - 43rd Annual Allerton Conference on Communication, Control and Computing 2005
Y2 - 28 September 2005 through 30 September 2005
ER -