Generalization of the de Bruijn Identity to General \phi -Entropies and \phi -Fisher Informations

• Published in: IEEE transactions on information theory Vol. 64; no. 10; pp. 6743 - 6758
• Main Authors: Valero-Toranzo, Irene, Zozor, Steeve, Brossier, Jean-Marc
• Format: Journal Article
• Language: English
• Published: IEEE 01.10.2018
• Subjects: <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">ϕ -entropy and <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">ϕ -divergences; <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">ϕ -Fisher information; Communication channels; Entropy; Estimation; Gaussian noise; generalized de Bruijn identities; Linear matrix inequalities; Mean square error methods; Mutual information; Probability density function
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2017.2771209

In this paper, we propose generalizations of the de Bruijn identity based on extensions of the Shannon entropy, Fisher information and their associated divergences or relative measures. The foundations of these generalizations are the <inline-formula> <tex-math notation="LaTeX">\phi </tex-math></inline-formula>-entropies and divergences of the Csiszár (or Salicrú) class considered within a multidimensional context, including the one-dimensional case, and for several types of noisy channels characterized by a more general probability distribution beyond the well-known Gaussian noise. We found that the gradient and/or the Hessian of these entropies or divergences with respect to the noise parameter naturally give rise to generalized versions of the Fisher information or divergence, which are named the <inline-formula> <tex-math notation="LaTeX">\phi </tex-math></inline-formula>-Fisher information (divergence). The obtained identities can be viewed as further extensions of the classical de Bruijn identity. Analogously, it is shown that a similar relation holds between the <inline-formula> <tex-math notation="LaTeX">\phi </tex-math></inline-formula>-divergence and an extended mean-square error, named <inline-formula> <tex-math notation="LaTeX">\phi </tex-math></inline-formula>-mean square error, for the Gaussian channel.