On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel

• Published in: IEEE transactions on information theory Vol. 51; no. 9; pp. 3017 - 3024
• Main Author: Zakai, M.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.09.2005; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Calculus; Channels; Communication channels; Comparative analysis; Derivation; Detection, estimation, filtering, equalization, prediction; Entropy; Errors; Estimation error; Exact sciences and technology; Gaussian; Gaussian channel; Gaussian channels; Indium tin oxide; Inequalities; Information filtering; Information filters; Information, signal and communications theory; Likelihood ratio; logarithmic Sobolev inequality; Malliavin calculus; Mathematical analysis; minimal mean-square estimation error; Mutual information; Noise measurement; nonlinear filtering; relative entropy; Signal and communications theory; Signal, noise; Telecommunications and information theory; White noise; Estimation error; Error estimation; Logarithmic function; Gaussian channel; Likelihood ratio; logarithmic Sobolev inequality; minimal mean-square estimation error; Entropy; Mean square error; Non linear filtering; Fisher information; Gaussian noise; relative entropy; White noise; nonlinear filtering; Causality; Malliavin calculus; Mutual information
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2005.853297

This paper considers the model of an arbitrarily distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean-square error of the noncausal estimator and the likelihood ratio between y and w are derived. This is followed by an extended version of a recently derived relation between the mutual information I(x;y) and the minimal mean-square error. These results are applied to derive infinite-dimensional versions of the Fisher information and the de Bruijn identity. A comparison between the causal and noncausal estimation errors yields a restricted form of the logarithmic Sobolev inequality. The derivation of the results is based on the Malliavin calculus