Neuromorphic computing with hybrid CNN–Stochastic Reservoir for time series WiFi based human activity recognition

• Published in: Computers & electrical engineering Vol. 111; p. 108917
• Main Authors: Saw, Chia Yee, Wong, Yan Chiew
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2023
• Subjects: Channel State Information; Convolution neural network; Human activity recognition; Neuromorphic Computing; Stochastic Reservoir; Template matching; Human activity recognition; Template matching; Channel State Information; Neuromorphic Computing; Convolution neural network; Stochastic Reservoir
• DOI: 10.1016/j.compeleceng.2023.108917
• ISSN: 0045-7906

Wi-Fi Channel State Information (CSI) based human activity recognition (HAR) which using channel disturbances caused by signal reflection is a novel way of environment sensing and motion recognition. The collected channels characteristics are heavily influenced by the environment, human activity patterns and subject’s weight and height. These signal variations reflected from body components are mainly affected by static multipath effects comprises random noise and behave differently in individuals, and thus an active field of research. To reach further for achieving automated real-time classification, lower computational cost and easy adaptability to hardware are necessary. In this work, a CSI-based HAR with hybrid framework, Convolutional Neural Network (CNN)-Stochastic Reservoir (SR) (CNN-SR) has been proposed, enabling a subject adaptable and more efficient hardware implementation with minimal computational complexity. A subcarrier correlation matrix is first computed and portrayed in image without preprocessing based on the reflection of the raw CSI signal induced by human activities at regular intervals, allowing visual observation of whole pattern changes. The time-based features are subsequently extracted through CNN and these feature arrays are then feed into SR which based on stochastic spiking neural network (SSNN) in simple cycle reservoir architecture for template matching. SR offers attractive power savings over typical von Neumann systems, by doing stochastic computations. The proposed method has also been demonstrated that is capable for HAR based on partially captured signals. The signal pattern of each segment can be observed in a single sight and then employed for person-to-person template recognition. This enables HAR with minimal computational complexity and solving the inter-person variability concerns. The results demonstrate that the proposed CNN-SR achieves impressive performance in recognizing human activities and surpasses existing models with an average accuracy of 93.49%. [Display omitted] •The proposed method using convolution neural network (CNN) to extract human activity features from complex CSI data without preprocessing, allowing visual observation of whole pattern changes.•Template matching implemented for activity recognition, allowing minimal computational complexity and enabling time series HAR which based on partially captured signals.•The proposed system robust in recognizing incomplete HAR signals and achieves impressive performance in recognizing human activities with an overall accuracy of 94.81%.