Shadow-based Hand Gesture Recognition in one Packet

• Published in: 2020 16th International Conference on Distributed Computing in Sensor Systems (DCOSS) pp. 27 - 34
• Main Authors: Hazra, Saptarshi, Brachmann, Martina, Voigt, Thiemo
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2020
• Subjects: Cloud computing; Data Acquisition; Deep Learning; Gesture recognition; IEEE 802.15 Standard; Image edge detection; Internet of Things (IoT); Machine learning; Microsoft Windows; Performance evaluation
• DOI: 10.1109/DCOSS49796.2020.00018

The ubiquity of wirelessly connected sensing devices in IoT applications provides the opportunity to enable various types of interaction with our digitally connected environment. Currently, low processing capabilities and high energy costs for communication limit the use of energy-constrained devices for this purpose. In this paper, we address this challenge by exploring the new possibilities highly capable deep neural network classifiers present. To reduce the energy consumption for transferring continuously sampled data, we propose to compress the sensed data and perform classification at the edge. We evaluate several compression methods in the context of a shadow-based hand gesture detection application, where the classification is performed using a convolutional neural network. We show that simple data reduction methods allow us to compress the sensed data into a single IEEE 802.15.4 packet while maintaining a classification accuracy of 93%. We further show the generality of our compression methods in an audio-based interaction scenario.