A Spiking Neuron as Information Bottleneck

• Published in: Neural computation Vol. 22; no. 8; pp. 1961 - 1992
• Main Authors: Buesing, Lars, Maass, Wolfgang
• Format: Journal Article
• Language: English
• Published: One Rogers Street, Cambridge, MA 02142-1209, USA MIT Press 01.08.2010; MIT Press Journals, The
• Subjects: Applied sciences; Artificial intelligence; Biological and medical sciences; Bottlenecks; Computer science; control theory; systems; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects; Information processing; Learning; Learning - physiology; Learning and adaptive systems; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects); Miscellaneous; Models, Neurological; Neuronal Plasticity - physiology; Neurons; Neurons - physiology; Relevance; Spiking neuron; Neural computation; Dimensionality; Neuronal discharge; Synaptic plasticity; Neural network; Relevant information; Information transmission; Information loss
• ISSN: 0899-7667; 1530-888X
• DOI: 10.1162/neco.2010.08-09-1084

Neurons receive thousands of presynaptic input spike trains while emitting a single output spike train. This drastic dimensionality reduction suggests considering a neuron as a bottleneck for information transmission. Extending recent results, we propose a simple learning rule for the weights of spiking neurons derived from the information bottleneck (IB) framework that minimizes the loss of relevant information transmitted in the output spike train. In the IB framework, relevance of information is defined with respect to contextual information, the latter entering the proposed learning rule as a “third” factor besides pre- and postsynaptic activities. This renders the theoretically motivated learning rule a plausible model for experimentally observed synaptic plasticity phenomena involving three factors. Furthermore, we show that the proposed IB learning rule allows spiking neurons to learn a predictive code, that is, to extract those parts of their input that are predictive for future input.