Human activity recognition with smartphone sensors using deep learning neural networks

• Published in: Expert systems with applications Vol. 59; pp. 235 - 244
• Main Authors: Ronao, Charissa Ann, Cho, Sung-Bae
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.10.2016
• Subjects: Classification; Convolutional neural network; Deep learning; Human activity recognition; Human motion; Invariants; Moving object recognition; Neural networks; Raw; Sensors; Smartphone; Smartphones; Deep learning; Sensors; Human activity recognition; Smartphone; Convolutional neural network
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2016.04.032

•This paper proposes a deep convolutional neural network for HAR using smartphone sensors.•Experiments show that the proposed method derives relevant and more complex features.•The method achieved an almost perfect classification on moving activities.•It outperforms other state-of-the-art data mining techniques in HAR. Human activities are inherently translation invariant and hierarchical. Human activity recognition (HAR), a field that has garnered a lot of attention in recent years due to its high demand in various application domains, makes use of time-series sensor data to infer activities. In this paper, a deep convolutional neural network (convnet) is proposed to perform efficient and effective HAR using smartphone sensors by exploiting the inherent characteristics of activities and 1D time-series signals, at the same time providing a way to automatically and data-adaptively extract robust features from raw data. Experiments show that convnets indeed derive relevant and more complex features with every additional layer, although difference of feature complexity level decreases with every additional layer. A wider time span of temporal local correlation can be exploited (1×9–1×14) and a low pooling size (1×2–1×3) is shown to be beneficial. Convnets also achieved an almost perfect classification on moving activities, especially very similar ones which were previously perceived to be very difficult to classify. Lastly, convnets outperform other state-of-the-art data mining techniques in HAR for the benchmark dataset collected from 30 volunteer subjects, achieving an overall performance of 94.79% on the test set with raw sensor data, and 95.75% with additional information of temporal fast Fourier transform of the HAR data set.