Minimization Problems Based on Relative \alpha -Entropy II: Reverse Projection

• Published in: IEEE transactions on information theory Vol. 61; no. 9; pp. 5081 - 5095
• Main Authors: Ashok Kumar, M., Sundaresan, Rajesh
• Format: Journal Article
• Language: English
• Published: IEEE 01.09.2015
• Subjects: Best approximant; Entropy; exponential family; information geometry; Kullback-Leibler divergence; linear family; Mathematical model; Maximum likelihood estimation; Minimization; Pollution measurement; power-law family; projection; Pythagorean property; Q measurement; relative entropy; Renyi entropy; robust estimation; Robustness; Tsallis entropy; information geometry; Rényi entropy; Tsallis entropy; robust estimation; relative entropy; power-law family; Best approximant; Pythagorean property; Kullback-Leibler divergence; linear family; projection; exponential family
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2015.2449312

In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted ℐ α ) were studied. Such minimizers were called forward ℐ α -projections. Here, a complementary class of minimization problems leading to the so-called reverse ℐ α -projections are studied. Reverse ℐ α -projections, particularly on log-convex or power-law families, are of interest in robust estimation problems (α > 1) and in constrained compression settings (α <; 1). Orthogonality of the power-law family with an associated linear family is first established and is then exploited to turn a reverse ℐ α -projection into a forward ℐ α -projection. The transformed problem is a simpler quasi-convex minimization subject to linear constraints.