Learning a Sparse Code for Temporal Sequences Using STDP and Sequence Compression

• Published in: Neural computation Vol. 23; no. 10; pp. 2567 - 2598
• Main Authors: Byrnes, Sean, Burkitt, Anthony N., Grayden, David B., Meffin, Hamish
• Format: Journal Article
• Language: English
• Published: One Rogers Street, Cambridge, MA 02142-1209, USA MIT Press 01.10.2011; MIT Press Journals, The
• Subjects: Applied sciences; Artificial intelligence; Biological and medical sciences; Brain; Codes; Computer science; control theory; systems; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects; Hippocampus - physiology; Inference from stochastic processes; time series analysis; Learning; Learning and adaptive systems; Letters; Mathematics; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects); Miscellaneous; Models, Neurological; Neural networks; Neural Networks (Computer); Neurons; Neurons - physiology; Probability and statistics; Sciences and techniques of general use; Statistics; Spiking neuron; Neural computation; Compression; Memory; Numerical model; Interference; Neural network; Learning; Depolarization; Synapse; Spike timing dependent plasticity; Plasticity; Numerical simulation; Simulation model; Hippocampus
• ISSN: 0899-7667; 1530-888X; 1530-888X
• DOI: 10.1162/NECO_a_00184

A spiking neural network that learns temporal sequences is described. A sparse code in which individual neurons represent sequences and subsequences enables multiple sequences to be stored without interference. The network is founded on a model of sequence compression in the hippocampus that is robust to variation in sequence element duration and well suited to learn sequences through spike-timing dependent plasticity (STDP). Three additions to the sequence compression model underlie the sparse representation: synapses connecting the neurons of the network that are subject to STDP, a competitive plasticity rule so that neurons specialize to individual sequences, and neural depolarization after spiking so that neurons have a memory. The response to new sequence elements is determined by the neurons that have responded to the previous subsequence, according to the competitively learned synaptic connections. Numerical simulations show that the model can learn sets of intersecting sequences, presented with widely differing frequencies, with elements of varying duration.