Minimax Rate-Distortion

• Published in: IEEE transactions on information theory Vol. 69; no. 12; pp. 7712 - 7737
• Main Authors: Mahmood, Adeel, Wagner, Aaron B.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Codes; Convergence; d-semifaithful code; Distortion; Distortion measurement; Entropy; Lossy compression; Minimax technique; Nonlinear distortion; quantization; Rate-distortion; Redundancy; Regularity; universal source coding; VC dimension
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2023.3328532

We show the existence of variable-rate rate-distortion codes that meet the distortion constraint almost surely and are minimax, i.e., strongly, universal with respect to an unknown source distribution and a distortion measure that is revealed only to the encoder and only at runtime. If we only require minimax universality with respect to the source distribution and not the distortion measure, then we provide an achievable <inline-formula> <tex-math notation="LaTeX">\tilde {O}(1/\sqrt {n}) </tex-math></inline-formula> redundancy rate, which we show is optimal. This is in contrast to prior work on universal lossy compression, which provides <inline-formula> <tex-math notation="LaTeX">O(\log n/n) </tex-math></inline-formula> redundancy guarantees for weakly universal codes under various regularity conditions. We show that either eliminating the regularity conditions or upgrading to strong universality while keeping these regularity conditions entails an inevitable increase in the redundancy to <inline-formula> <tex-math notation="LaTeX">\tilde {O}(1/\sqrt {n}) </tex-math></inline-formula>. Our construction involves random coding with non-i.i.d. codewords and a zero-rate uncoded transmission scheme. The proof uses exact asymptotics from large deviations, acceptance-rejection sampling, and the VC dimension of distortion measures.