Lossy Compression of Discrete Sources via the Viterbi Algorithm

• Published in: IEEE transactions on information theory Vol. 58; no. 4; pp. 2475 - 2489
• Main Authors: Jalali, S., Montanari, A., Weissman, T.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2012; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Applied sciences; Approximation; Approximation algorithms; Approximation methods; Codes; Coding, codes; Compression algorithms; Entropy; Exact sciences and technology; Information theory; Information, signal and communications theory; Rate-distortion coding; Signal and communications theory; Telecommunications and information theory; universal compression; Vectors; Viterbi algorithm; Alphabet; Lossy compression; Performance evaluation; Cost minimization; Lossless circuit; Coding circuit; Rate distortion theory; Ergodicity; Lossy medium; Iterative method; universal compression; Decoding; Algorithm; Viterbi code; Viterbi algorithm; Linear combination; Coding; Rate-distortion coding; Cost function
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2011.2178059

We present a new lossy compressor for finite-alphabet sources. For coding a sequence x n , the encoder starts by assigning a certain cost to each possible reconstruction sequence. It then finds the one that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of each sequence is a linear combination of its distance from the sequence x n and a linear function of its k th order empirical distribution. The structure of the cost function allows the encoder to employ the Viterbi algorithm to find the sequence with minimum cost. We identify a choice of the coefficients used in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance for any stationary ergodic source, in the limit of large , provided that increases as o(log n). Iterative techniques for approximating the coefficients, which alleviate the computational burden of finding the optimal coefficients, are proposed and studied.