A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing

Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implement...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Meyer, Svea Marie, Weidel, Philipp, Plank, Philipp, Campos-Macias, Leobardo, Sumit Bam Shrestha, Stratmann, Philipp, Richter, Mathis
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 23.09.2024
Subjects
Microprocessors
Neuromorphic computing
Real time
State space models
Online AccessGet full text

Cover

More Information
Summary:Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.
ISSN:2331-8422