BNN An Ideal Architecture for Acceleration With Resistive in Memory Computation

• Published in: IEEE transactions on emerging topics in computing Vol. 11; no. 2; pp. 281 - 291
• Main Authors: Ding, Andrew, Qiao, Ye, Bagherzadeh, Nader
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Acceleration; Accelerators; Arrays; Binary neural network; Computational modeling; Computer architecture; Computing time; deep learning; ECRAM; Feature maps; Hardware; in memory processing; Machine learning; Mathematical analysis; Memory devices; Memristors; Neural networks; Noise tolerance; PCM; Performance evaluation; Power consumption; Power management; Programming; RRAM
• ISSN: 2168-6750; 2168-6750
• DOI: 10.1109/TETC.2023.3237778

Binary Neural Networks (BNN) have binarized (-1 and 1) weights and feature maps. Achieving smaller model sizes and computational simplicity, they are well suited for edge-AI systems with power and hardware constraints. Recently, memristive crossbar arrays have gained considerable attention from researchers to perform analog in-memory vector-matrix multiplications in machine learning accelerators, with low power and constant computational time. Crossbar arrays suffer from many non-ideal characteristics such as memristor device imperfections, weight noise, device drift, input/output noises, and DAC/ADC overhead. Thus, for analog AI acceleration to become viable, model architectures must be robust against these non-idealities. We propose that BNN's with their binarized weights, which are ideally mapped to fewer memristive devices with less electrical characteristic issues and higher tolerance to computational noise, are a promising architecture for analog computation. In this work, we examine the viability of deploying state of the art BNNs, with features such as real value residual connections and parametric activations with biases, to analog in-memory computational accelerators. Our simulations show that BNNs are significantly more robust to crossbar non-idealities than full-precision networks, require less chip area, and consume less power on memristive crossbar architectures.