Strong Functional Representation Lemma and Applications to Coding Theorems

• Published in: IEEE transactions on information theory Vol. 64; no. 11; pp. 6967 - 6978
• Main Authors: Li, Cheuk Ting, Gamal, Abbas El
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Channel coding; channel simulation; channel with state; Coding; Continuity (mathematics); Electrical engineering; Functional representation lemma; Indexes; lossy source coding; one-shot achievability; Random variables; Representations; Source coding; Theorems
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2018.2865570

This paper shows that for any random variables <inline-formula> <tex-math notation="LaTeX">X </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">Y </tex-math></inline-formula>, it is possible to represent <inline-formula> <tex-math notation="LaTeX">Y </tex-math></inline-formula> as a function of <inline-formula> <tex-math notation="LaTeX">(X,Z) </tex-math></inline-formula> such that <inline-formula> <tex-math notation="LaTeX">Z </tex-math></inline-formula> is independent of <inline-formula> <tex-math notation="LaTeX">X </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">I(X;Z|Y)\le \log (I(X;Y)+1)+4 </tex-math></inline-formula> bits. We use this strong functional representation lemma (SFRL) to establish a bound on the rate needed for one-shot exact channel simulation for general (discrete or continuous) random variables, strengthening the results by Harsha et al. and Braverman and Garg, and to establish new and simple achievability results for one-shot variable-length lossy source coding, multiple description coding, and Gray-Wyner system. We also show that the SFRL can be used to reduce the channel with state noncausally known at the encoder to a point-to-point channel, which provides a simple achievability proof of the Gelfand-Pinsker theorem.