Quantization Framework for Fast Spiking Neural Networks

• Published in: Frontiers in neuroscience Vol. 16; p. 918793
• Main Authors: Li, Chen, Ma, Lei, Furber, Steve
• Format: Journal Article
• Language: English
• Published: Frontiers Media S.A 19.07.2022
• Subjects: ANN-to-SNN conversion; fast spiking neural networks; inference latency; Neuroscience; occasional noise; quantization; spiking neural networks
• DOI: 10.3389/fnins.2022.918793
• ISSN: 1662-453X; 1662-4548

Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) offer additional temporal dynamics with the compromise of lower information transmission rates through the use of spikes. When using an ANN-to-SNN conversion technique there is a direct link between the activation bit precision of the artificial neurons and the time required by the spiking neurons to represent the same bit precision. This implicit link suggests that techniques used to reduce the activation bit precision of ANNs, such as quantization, can help shorten the inference latency of SNNs. However, carrying ANN quantization knowledge over to SNNs is not straightforward, as there are many fundamental differences between them. Here we propose a quantization framework for fast SNNs (QFFS) to overcome these difficulties, providing a method to build SNNs with enhanced latency and reduced loss of accuracy relative to the baseline ANN model. In this framework, we promote the compatibility of ANN information quantization techniques with SNNs, and suppress “occasional noise” to minimize accuracy loss. The resulting SNNs overcome the accuracy degeneration observed previously in SNNs with a limited number of time steps and achieve an accuracy of 70.18% on ImageNet within 8 time steps. This is the first demonstration that SNNs built by ANN-to-SNN conversion can achieve a similar latency to SNNs built by direct training.