Binarized SNNs: Efficient and Error-Resilient Spiking Neural Networks through Binarization

• Published in: 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD) pp. 1 - 9
• Main Authors: Wei, Ming-Liang, Yayla, Mikail, Ho, Shu-Yin, Chen, Jian-Jia, Yang, Chia-Lin, Amrouch, Hussam
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2021
• Subjects: Artificial neural networks; Capacitors; Computational modeling; Costs; Firing; Stochastic processes; Time-frequency analysis
• ISSN: 1558-2434
• DOI: 10.1109/ICCAD51958.2021.9643463

Spiking Neural Networks (SNNs) are considered the third generation of NNs and can reach similar accuracy as conventional deep NNs, but with a considerable improvement in efficiency. However, to achieve high accuracy, state-of-the-art SNNs employ stochastic spike coding of the inputs, requiring multiple cycles of computation. Because of this and due to the nature of analog computing, it is required to accumulate and hold the charges of multiple cycles, necessitating a large membrane capacitor. This results in high energy, long latency, and expensive area costs, constituting one of the major bottlenecks in analog SNN implementations. Membrane capacitor size determines the precision of the firing time. Hence reducing the capacitor size considerably degrades the inference accuracy. To alleviate this, we focus on bridging the gap between binarized NNs (BNNs) and SNNs. BNNs are rapidly emerging as an attractive alternative for NNs due to their high efficiency and error tolerance. In this work, we evaluate the impact of deploying error-resilient BNNs, i.e. BNNs that have been proactively trained in the presence of errors, on analog implementation of SNNs. We show that for BNNs, the capacitor size and latency can be reduced significantly compared to state-of-the-art SNNs, which employ multi-bit models. Our experiments demonstrate that when error-resilient BNNs are deployed on analog-based SNN accelerator, the size of the membrane capacitor is reduced by 50%, the inference latency is decreased by two orders of magnitude, and energy is reduced by 57% compared to the baseline 4-bit SNN implementation, under minimal accuracy cost.