Performance comparison of DVS data spatial downscaling methods using Spiking Neural Networks

• Published in: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) pp. 6483 - 6491
• Main Authors: Gruel, Amelie, Martinet, Jean, Linares-Barranco, Bernabe, Serrano-Gotarredona, Teresa
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2023
• Subjects: and algorithms (including transfer; and un-supervised learning; Applications: Robotics; Computer vision; Embedded sensing/real-time techniques; Embedded systems; formulations; low-shot; Machine learning architectures; Neural networks; Neuromorphics; Real-time systems; Robot vision systems; self; semi; Vision sensors
• ISSN: 2642-9381
• DOI: 10.1109/WACV56688.2023.00643

Dynamic Vision Sensors (DVS) are an unconventional type of camera that produces sparse and asynchronous event data, which has recently led to a strong increase in its use for computer vision tasks namely in robotics. Embedded systems face limitations in terms of energy resources, memory, computational power, and communication bandwidth. Hence, this application calls for a way to reduce the amount of data to be processed while keeping the relevant information for the task at hand. We thus believe that a formal definition of event data reduction methods will provide a step further towards sparse data processing.The contributions of this paper are twofold: we introduce two complementary neuromorphic methods based on Spiking Neural Networks for DVS data spatial reduction, which is to best of our knowledge the first proposal of neuromorphic event data reduction; then we study for each method the trade-off between the amount of information kept after reduction, the performance of gesture classification after reduction and their capacity to handle events in real time. We demonstrate here that the proposed SNN-based methods outperform existing methods in a classification task for most dividing factors and are significantly better at handling data in real time, and make therefore the optimal choice for fully-integrated energy-efficient event data reduction running dynamically on a neuromorphic platform. Our code is publicly available online at: https://github.com/amygruel/EvVisu.