Cellular Automata Can Reduce Memory Requirements of Collective-State Computing

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 6; pp. 2701 - 2713
• Main Authors: Kleyko, Denis, Frady, Edward Paxon, Sommer, Friedrich T.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Automata; Automaton; Cellular automata; Cellular automata (CA); Cellular automaton; Cellular automatons; collective-state computing; Computational modeling; Computational modelling; Computer memory; Data processing; Decoding; Distributed representation; distributed representations; hyperdimensional computing; Information processing; Job analysis; Memory architecture; Memory management; Network architecture; Neural networks; Neurons; Pseudorandom; Random processes; Random-number generation; Randomization; Representations; Reservoir Computing; reservoir computing (RC); Reservoir management; Reservoirs; rule 90; State vectors; Task analysis; Vector symbolic architecture; vector symbolic architectures (VSAs)
• DOI: 10.1109/TNNLS.2021.3119543
• ISSN: 2162-237X; 2162-2388

Various nonclassical approaches of distributed information processing, such as neural networks, reservoir computing (RC), vector symbolic architectures (VSAs), and others, employ the principle of collective-state computing. In this type of computing, the variables relevant in computation are superimposed into a single high-dimensional state vector, the collective state. The variable encoding uses a fixed set of random patterns, which has to be stored and kept available during the computation. In this article, we show that an elementary cellular automaton with rule 90 (CA90) enables the space-time tradeoff for collective-state computing models that use random dense binary representations, i.e., memory requirements can be traded off with computation running CA90. We investigate the randomization behavior of CA90, in particular, the relation between the length of the randomization period and the size of the grid, and how CA90 preserves similarity in the presence of the initialization noise. Based on these analyses, we discuss how to optimize a collective-state computing model, in which CA90 expands representations on the fly from short seed patterns-rather than storing the full set of random patterns. The CA90 expansion is applied and tested in concrete scenarios using RC and VSAs. Our experimental results show that collective-state computing with CA90 expansion performs similarly compared to traditional collective-state models, in which random patterns are generated initially by a pseudorandom number generator and then stored in a large memory.