Real-time walking gait terrain classification from foot-mounted Inertial Measurement Unit using Convolutional Long Short-Term Memory neural network

• Published in: Expert systems with applications Vol. 203; p. 117306
• Main Authors: Moura Coelho, Rui, Gouveia, João, Botto, Miguel Ayala, Krebs, Hermano Igo, Martins, Jorge
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2022
• Subjects: Convolutional Neural Network; Gait terrain; Inertial Measurement Unit; Long Short-Term Memory; Real-time classification; Convolutional Neural Network; Gait terrain; Real-time classification; Long Short-Term Memory; Inertial Measurement Unit
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.117306

We propose a novel online real-time gait terrain detection algorithm from the measurements of a foot-mounted Inertial Measurement Unit (IMU), using a shallow cascaded Convolutional and Long Short-Term Memory neural network (CNN-LSTM). Gait data is acquired from healthy subjects walking in an unstructured environment that includes level ground, stair ascent and stair descent. The CNN-LSTM subject-independent classifier is trained to continuously detect the terrain from the time series data, invariant to IMU initial pose. Our results show that the classifier is able to correctly detect the terrain on data from unseen subjects, in less than 90ms from toe-off (f1-score >0.89), improving further its classification performance in less than 135ms from toe-off (f1-score >0.98). Furthermore, we present a novel capability with this classifier to timely detect terrain transitions, switching from the starting to the final terrain during midswing. The CNN-LSTM classifier is therefore suitable to be used in assistive devices, timely adjusting to the different gait kinematics, using a single foot-mounted IMU. •Real-Time Gait terrain classification from foot IMU in <135ms after push-off.•Capability to timely detect transitions between terrains in midswing.•Subject-independent classifier.•Invariant to sensor orientation.•Shallow CNN-LSTM network suitable for implementation in low-cost hardware.