Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

• Published in: Sensors (Basel, Switzerland) Vol. 20; no. 12; p. 3404
• Main Authors: Tapiador-Morales, Ricardo, Maro, Jean-Matthieu, Jimenez-Fernandez, Angel, Jimenez-Moreno, Gabriel, Benosman, Ryad, Linares-Barranco, Alejandro
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 16.06.2020; MDPI
• Subjects: Algorithms; Change detection; Computer architecture; dynamic vision sensors; Embedded systems; event-based; Field programmable gate arrays; FPGA; Gesture recognition; HDL; Luminosity; Network latency; Neural networks; Neurons; Pattern recognition; Power; Prototypes; Recognition; Sensors; synchronous digital VLSI; Temporal resolution; Vision; HDL; FPGA; event-based; dynamic vision sensors; synchronous digital VLSI; AER; pattern recognition
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s20123404

Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1 μ s to 6.7 μ s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.