Multimodal Sensor Fusion for Low-Power Wearable Human Motion Tracking Systems in Sports Applications

• Published in: IEEE sensors journal Vol. 21; no. 4; pp. 5195 - 5212
• Main Authors: Wilk, Mariusz P., Walsh, Michael, O'Flynn, Brendan
• Format: Journal Article
• Language: English
• Published: New York IEEE 15.02.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3-D; Algorithms; Cameras; Human motion; Inertial motion sensor; Inertial sensing devices; Injuries; inside-out tracking; monocular camera; Motion sensors; multimodal; Multimodal sensors; Physical training; pose detection; Prototypes; Root-mean-square errors; Sensor fusion; Sensors; Sports; Tracking; Tracking systems; Vertical motion; Vision sensors; Wearable technology
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2020.3030779

This paper presents a prototype human motion tracking system for wearable sports applications. It can be particularly applicable for tracking human motion during executing certain strength training exercises, such as the barbell squat, where an inappropriate technique could result in an injury. The key novelty of the proposed system is twofold. Firstly, it is an inside-out, multimodal, motion tracker that incorporates two complementary sensor modalities, i.e. a camera and an inertial motion sensor, as well as two externally-mounted points of reference. Secondly, it incorporates a novel multimodal sensor fusion algorithm which uses the complementary nature of vision and inertial sensor modalities to perform a computationally efficient 3-Dimensional (3-D) pose detection of the wearable device. The 3-D pose is determined by fusing information about the two external reference points captured by the camera together with the orientation angles captured by the inertial motion sensor. The accuracy of the prototype was experimentally validated in laboratory conditions. The main findings are as follows. The Root Mean Square Error (RMSE) in 3-D position calculation was 36.7 mm and 13.6 mm in the static and mobile cases, respectively. Whereas the static case was aimed at determining the system's performance at all 3-D poses within the work envelope, the mobile case was used to determine the error in tracking human motion that is involved in the barbell squat, i.e. a mainly repeated vertical motion pattern.