Sparse sign-consistent Johnson–Lindenstrauss matrices: Compression with neuroscience-based constraints

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 111; no. 47; pp. 16872 - 16876
• Main Authors: Allen-Zhu, Zeyuan, Gelashvili, Rati, Micali, Silvio, Shavit, Nir
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 25.11.2014; National Acad Sciences
• Subjects: Approximation; Biological neural networks; Biological Sciences; Brain; Correlation analysis; Dimensionality; Eigenvalues; Mathematical vectors; Matrices; Matrix; Models, Biological; Neurons; Neuroscience; Neurosciences; Physical Sciences; Tissues; Vertices; Johnson–Lindenstrauss compression; synaptic-connectivity matrices; sign-consistent matrices
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.1419100111

Significance Significant biological evidence indicates that the brain may perform some form of compression. To be meaningful, such compression should preserve pairwise correlation of the input data. It is mathematically well known that multiplying the input vectors by a sparse and fixed random matrix A achieves the desired compression. But, to implement such an approach in the brain via a synaptic-connectivity matrix, A should also to be sign consistent: that is, all entries in a single column must be either all nonnegative or all nonpositive. This is so because most neurons are either excitatory or inhibitory. We prove that sparse sign-consistent matrices can deliver the desired compression, lending credibility to the hypothesis that correlation-preserving compression occurs in the brain via synaptic-connectivity matrices. Johnson–Lindenstrauss (JL) matrices implemented by sparse random synaptic connections are thought to be a prime candidate for how convergent pathways in the brain compress information. However, to date, there is no complete mathematical support for such implementations given the constraints of real neural tissue. The fact that neurons are either excitatory or inhibitory implies that every so implementable JL matrix must be sign consistent (i.e., all entries in a single column must be either all nonnegative or all nonpositive), and the fact that any given neuron connects to a relatively small subset of other neurons implies that the JL matrix should be sparse . We construct sparse JL matrices that are sign consistent and prove that our construction is essentially optimal. Our work answers a mathematical question that was triggered by earlier work and is necessary to justify the existence of JL compression in the brain and emphasizes that inhibition is crucial if neurons are to perform efficient, correlation-preserving compression.