Noise-Resilient DNN: Tolerating Noise in PCM-Based AI Accelerators via Noise-Aware Training

• Published in: IEEE transactions on electron devices Vol. 68; no. 9; pp. 4356 - 4362
• Main Authors: Kariyappa, Sanjay, Tsai, Hsinyu, Spoon, Katie, Ambrogio, Stefano, Narayanan, Pritish, Mackin, Charles, Chen, An, Qureshi, Moinuddin, Burr, Geoffrey W.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accelerators; AI hardware; analog accelerators; Artificial neural networks; Computational modeling; Computer memory; Degradation; Direct reduction; Image classification; Model accuracy; Neural networks; Neurons; nonvolatile memory; Phase change materials; phase change memory (PCM); Programming; Recurrent neural networks; Regularization; Resilience; RRAM; Training
• ISSN: 0018-9383; 1557-9646
• DOI: 10.1109/TED.2021.3089987

Phase change memory (PCM)-based "Analog-AI" accelerators are gaining importance for inference in edge applications due to the energy efficiency offered by in-memory computing. Nevertheless, noise sources inherent to PCM devices cause inaccuracies in the deep neural network (DNN) weight values. Such inaccuracies can lead to severe degradation in model accuracy. To address this, we propose two techniques to improve noise resiliency of DNNs: 1) drift regularization (DR) and 2) multiplicative noise training (MNT). We evaluate convolutional networks trained on image classification and recurrent neural networks trained on language modeling and show that our techniques improve model accuracy by up to 12% over one month.