Optimised weight programming for analogue memory-based deep neural networks

• Published in: Nature communications Vol. 13; no. 1; p. 3765
• Main Authors: Mackin, Charles, Rasch, Malte J., Chen, An, Timcheck, Jonathan, Bruce, Robert L., Li, Ning, Narayanan, Pritish, Ambrogio, Stefano, Le Gallo, Manuel, Nandakumar, S. R., Fasoli, Andrea, Luquin, Jose, Friz, Alexander, Sebastian, Abu, Tsai, Hsinyu, Burr, Geoffrey W.
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 30.06.2022; Nature Publishing Group; Nature Portfolio
• Subjects: 639/166/987; 639/705/1041; Accuracy; Artificial neural networks; Complexity; Computer applications; Graphics processing units; Hardware; Humanities and Social Sciences; Inference; Memory devices; Mental task performance; multidisciplinary; Neural networks; Science; Science (multidisciplinary); Software
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-022-31405-1

Analogue memory-based deep neural networks provide energy-efficiency and per-area throughput gains relative to state-of-the-art digital counterparts such as graphics processing units. Recent advances focus largely on hardware-aware algorithmic training and improvements to circuits, architectures, and memory devices. Optimal translation of software-trained weights into analogue hardware weights—given the plethora of complex memory non-idealities—represents an equally important task. We report a generalised computational framework that automates the crafting of complex weight programming strategies to minimise accuracy degradations during inference, particularly over time. The framework is agnostic to network structure and generalises well across recurrent, convolutional, and transformer neural networks. As a highly flexible numerical heuristic, the approach accommodates arbitrary device-level complexity, making it potentially relevant for a variety of analogue memories. By quantifying the limit of achievable inference accuracy, it also enables analogue memory-based deep neural network accelerators to reach their full inference potential. Device-level complexity represents a big shortcoming for the hardware realization of analogue memory-based deep neural networks. Mackin et al. report a generalized computational framework, translating software-trained weights into analogue hardware weights, to minimise inference accuracy degradation.