Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 10; pp. 2408 - 2421
• Main Authors: Soudry, Daniel, Di Castro, Dotan, Gal, Asaf, Kolodny, Avinoam, Kvatinsky, Shahar
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Backpropagation; Circuits; CMOS; Computer Simulation; Descent; Hardware; Humans; Learning; Memory devices; memristive systems; memristor; Memristors; multilayer neural networks (MNNs); Multilayers; Neural networks; Neural Networks (Computer); Neurons - physiology; Online Systems - instrumentation; Signal Processing, Computer-Assisted - instrumentation; stochastic gradient descent; synapse; Synapses - physiology; Training; Transistors; Backpropagation; multilayer neural networks (MNNs); memristor; synapse; memristive systems; stochastic gradient descent; hardware

Learning in multilayer neural networks (MNNs) relies on continuous updating of large matrices of synaptic weights by local rules. Such locality can be exploited for massive parallelism when implementing MNNs in hardware. However, these update rules require a multiply and accumulate operation for eac...