Vector quantization by deterministic annealing

• Published in: IEEE transactions on information theory Vol. 38; no. 4; pp. 1249 - 1257
• Main Authors: Rose, K., Gurewitz, E., Fox, G.C.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.07.1992; Institute of Electrical and Electronics Engineers
• Subjects: Applied sciences; Design optimization; Entropy; Exact sciences and technology; Information, signal and communications theory; Lagrangian functions; Process control; Sampling, quantization; Signal and communications theory; Simulated annealing; Stochastic processes; Telecommunications and information theory; Temperature distribution; Training data; Vector quantization; Cluster; Lagrange multiplier; Annealing; Method of maximum entropy; Vector quantization; Computer simulation
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/18.144705

A deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data. The problem is reformulated within a probabilistic framework. No prior knowledge is assumed on the source density, and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion. The corresponding Lagrange multiplier is inversely related to the 'temperature' and is used to control the annealing process. In this process, as the temperature is lowered, the system undergoes a sequence of phase transitions when existing clusters split naturally, without use of heuristics. The resulting codebook is independent of the codebook used to initialize the iterations.< >