Sharp variance-entropy comparison for nonnegative Gaussian quadratic forms

In this article we study weighted sums of $n$ i.i.d. Gamma($\alpha$) random variables with nonnegative weights. We show that for $n \geq 1/\alpha$ the sum with equal coefficients maximizes differential entropy when variance is fixed. As a consequence, we prove that among nonnegative quadratic forms...

Full description

Saved in:
Bibliographic Details
Main Authors Bartczak, Maciej, Nayar, Piotr, Zwara, Szymon
Format Journal Article
LanguageEnglish
Published 24.05.2020
Subjects
Computer Science - Information Theory
Mathematics - Information Theory
Mathematics - Probability
Online AccessGet full text

Cover

More Information
Summary:In this article we study weighted sums of $n$ i.i.d. Gamma($\alpha$) random variables with nonnegative weights. We show that for $n \geq 1/\alpha$ the sum with equal coefficients maximizes differential entropy when variance is fixed. As a consequence, we prove that among nonnegative quadratic forms in $n$ independent standard Gaussian random variables, a diagonal form with equal coefficients maximizes differential entropy, under a fixed variance. This provides a sharp lower bound for the relative entropy between a nonnegative quadratic form and a Gaussian random variable. Bounds on capacities of transmission channels subject to $n$ independent additive gamma noises are also derived.
DOI:10.48550/arxiv.2005.11705