Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation

• Published in: IEEE transactions on information theory Vol. 63; no. 3; pp. 1580 - 1600
• Main Authors: Aolin Xu, Raginsky, Maxim
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bayes risk; Bayesian analysis; Channels; Communication; Communication channels; Data processing; decentralized estimation; Distortion; Entropy; Estimation; Information theory; Lower bounds; Microprocessors; Minimax technique; Mutual information; Neyman-Pearson converse; Processors; Program processors; Risk; Risk assessment; small ball probability; strong data processing inequalities
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2016.2646342

We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from local processors that observe the samples. The received data are subject to communication constraints due to the quantization and the noisy communication channels from the processors to the estimator. We first derive general lower bounds on the Bayes risk using information-theoretic quantities, such as mutual information, information density, small ball probability, and differential entropy. We then apply these lower bounds to the decentralized case, using strong data processing inequalities to quantify the contraction of information due to communication constraints. We treat the cases of a single processor and of multiple processors, where the samples observed by different processors may be conditionally dependent given the parameter, for noninteractive and interactive communication protocols. Our results recover and improve recent lower bounds on the Bayes risk and the minimax risk for certain decentralized estimation problems, where previously only conditionally independent sample sets and noiseless channels have been considered. Moreover, our results provide a general way to quantify the degradation of estimation performance caused by distributing resources to multiple processors, which is only discussed for specific examples in existing works.