TY - GEN

T1 - On the Structure of the Least Favorable Prior Distributions

AU - Dytso, Alex

AU - Poor, H. Vincent

AU - Bustin, Ronit

AU - Shamai, Shlomo

N1 - Funding Information:
The work of A. Dytso and H. V. Poor was supported by the U. S. National Science Foundation under Grants CCF-1420575, ECCS-1549881 and EECS-1647198. The work of S. Shamai and R. Bustin was supported by the European Union’s Horizon 2020 Research and Innovation Programme Grant 694630.
Publisher Copyright:
© 2018 IEEE.

PY - 2018/8/15

Y1 - 2018/8/15

N2 - This paper studies optimization of the minimum mean square error (MMSE) in order to characterize the structure of the least favorable prior distributions. In the first part, the paper characterizes the local behavior of the MMSE in terms of the input distribution and finds the directional derivative of the MMSE at the distribution P-{\mathbf{X}} in the direction of the distribution Q-{\mathbf{X}}. In the second part of the paper, the directional derivative together with the theory of convex optimization is used to characterize the structure of least favorable distributions. In particular, under mild regularity conditions, it is shown that the support of the least favorable distributions must necessarily be very small and is contained in a nowhere dense set of Lebesgue measure zero. The results of this paper produce both sufficient and necessary conditions for optimality, do not rely on Gaussian statistics assumptions, and are not sensitive to the dimensionality of random vectors. The results are evaluated for the univariate and multivariate random Gaussian cases, and the Poisson case. Finally, as one of the applications, it is shown how the results can be used to characterize the capacity of Gaussian MIMO channels with an amplitude constraint.

AB - This paper studies optimization of the minimum mean square error (MMSE) in order to characterize the structure of the least favorable prior distributions. In the first part, the paper characterizes the local behavior of the MMSE in terms of the input distribution and finds the directional derivative of the MMSE at the distribution P-{\mathbf{X}} in the direction of the distribution Q-{\mathbf{X}}. In the second part of the paper, the directional derivative together with the theory of convex optimization is used to characterize the structure of least favorable distributions. In particular, under mild regularity conditions, it is shown that the support of the least favorable distributions must necessarily be very small and is contained in a nowhere dense set of Lebesgue measure zero. The results of this paper produce both sufficient and necessary conditions for optimality, do not rely on Gaussian statistics assumptions, and are not sensitive to the dimensionality of random vectors. The results are evaluated for the univariate and multivariate random Gaussian cases, and the Poisson case. Finally, as one of the applications, it is shown how the results can be used to characterize the capacity of Gaussian MIMO channels with an amplitude constraint.

UR - http://www.scopus.com/inward/record.url?scp=85048567775&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048567775&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2018.8437546

DO - 10.1109/ISIT.2018.8437546

M3 - Conference contribution

AN - SCOPUS:85048567775

SN - 9781538647806

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 1081

EP - 1085

BT - 2018 IEEE International Symposium on Information Theory, ISIT 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE International Symposium on Information Theory, ISIT 2018

Y2 - 17 June 2018 through 22 June 2018

ER -