MSAT: Biologically Inspired Multi-Stage Adaptive Threshold for Conversion of Spiking Neural Networks
Spiking Neural Networks (SNNs) can do inference with low power consumption due to their spike sparsity. ANN-SNN conversion is an efficient way to achieve deep SNNs by converting well-trained Artificial Neural Networks (ANNs). However, the existing methods commonly use constant threshold for conversi...
Saved in:
Main Authors | , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
23.03.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Spiking Neural Networks (SNNs) can do inference with low power consumption
due to their spike sparsity. ANN-SNN conversion is an efficient way to achieve
deep SNNs by converting well-trained Artificial Neural Networks (ANNs).
However, the existing methods commonly use constant threshold for conversion,
which prevents neurons from rapidly delivering spikes to deeper layers and
causes high time delay. In addition, the same response for different inputs may
result in information loss during the information transmission. Inspired by the
biological model mechanism, we propose a multi-stage adaptive threshold (MSAT).
Specifically, for each neuron, the dynamic threshold varies with firing history
and input properties and is positively correlated with the average membrane
potential and negatively correlated with the rate of depolarization. The
self-adaptation to membrane potential and input allows a timely adjustment of
the threshold to fire spike faster and transmit more information. Moreover, we
analyze the Spikes of Inactivated Neurons error which is pervasive in early
time steps and propose spike confidence accordingly as a measurement of
confidence about the neurons that correctly deliver spikes. We use such spike
confidence in early time steps to determine whether to elicit spike to
alleviate this error. Combined with the proposed method, we examine the
performance on non-trivial datasets CIFAR-10, CIFAR-100, and ImageNet. We also
conduct sentiment classification and speech recognition experiments on the IDBM
and Google speech commands datasets respectively. Experiments show
near-lossless and lower latency ANN-SNN conversion. To the best of our
knowledge, this is the first time to build a biologically inspired multi-stage
adaptive threshold for converted SNN, with comparable performance to
state-of-the-art methods while improving energy efficiency. |
---|---|
DOI: | 10.48550/arxiv.2303.13080 |