Metabolically Efficient Information Processing

• Published in: Neural computation Vol. 13; no. 4; pp. 799 - 815
• Main Authors: Balasubramanian, Vijay, Kimber, Don, II, Michael J. Berry
• Format: Journal Article
• Language: English
• Published: One Rogers Street, Cambridge, MA 02142-1209, USA MIT Press 01.04.2001
• Subjects: Action Potentials; Algorithms; Applied sciences; Energy Metabolism; Exact sciences and technology; Information Theory; Information, signal and communications theory; Models, Neurological; Retinal Ganglion Cells - physiology; Synaptic Transmission; Telecommunications and information theory; Correlation; Noise; Transmission channel; Entropy; Invariance; Metabolically constrained capacity; Optimization; Communication theory; Information measure; Neurology; Energy; Coding; Efficiency; Information processing; Arimoto Blahut algorithm; Transmission rate; Energy efficient coding; Information theory; Neurobiological system
• ISSN: 0899-7667; 1530-888X
• DOI: 10.1162/089976601300014358

Energy-efficient information transmission may be relevant to biological sensory signal processing as well as to low-power electronic devices. We explore its consequences in two different regimes. In an “immediate” regime, we argue that the information rate should be maximized subject to a power constraint, and in an “exploratory” regime, the transmission rate per power cost should be maximized. In the absence of noise, discrete inputs are optimally encoded into Boltzmann distributed output symbols. In the exploratory regime, the partition function of this distribution is numerically equal to 1. The structure of the optimal code is strongly affected by noise in the transmission channel. The Arimoto-Blahut algorithm, generalized for cost constraints, can be used to derive and interpret the distribution of symbols for optimal energy-efficient coding in the presence of noise. We outline the possibilities and problems in extending our results to information coding and transmission in neurobiological systems.