Decentralized Data Reduction With Quantization Constraints

• Published in: IEEE transactions on signal processing Vol. 62; no. 7; pp. 1775 - 1784
• Main Authors: Xu, Ge, Zhu, Shengyu, Chen, Biao
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2014; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Bayes methods; Data compression; Data reduction; Decentralization; Decentralized; Decentralized inference; Decision making; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Inference; Information, signal and communications theory; Markov processes; Materials; Mathematical analysis; Optimization; Quantization; Quantization (signal); Sampling, quantization; Sensor fusion; Signal and communications theory; Signal, noise; Statistics; sufficiency principle; sufficient statistic; Telecommunications and information theory; Transaction processing; sufficient statistic; Decentralized inference; sufficiency principle; Decentralized system; Signal quantization; Measurement sensor; Signal processing; Implementation; Data reduction; quantization
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2014.2303432

A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impact of quantization on decentralized data reduction, which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.