A survey of depth and inertial sensor fusion for human action recognition

• Published in: Multimedia tools and applications Vol. 76; no. 3; pp. 4405 - 4425
• Main Authors: Chen, Chen, Jafari, Roozbeh, Kehtarnavaz, Nasser
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2017; Springer Nature B.V
• Subjects: 3-D graphics; Algorithms; Camcorders; Cameras; Computer Communication Networks; Computer engineering; Computer Science; Data Structures and Information Theory; Datasets; Energy consumption; Human body; Human motion; Inertial sensing devices; Lighting; Motion capture; Motion detectors; Moving object recognition; Multimedia; Multimedia computer applications; Multimedia Information Systems; Sensors; Special Purpose and Application-Based Systems; Studies; Vision; Vision systems; United States > US; Human action recognition; Depth sensor; Activity recognition; Inertial sensor; Sensor fusion; Multimodal dataset; 3D action data
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-015-3177-1

A number of review or survey articles have previously appeared on human action recognition where either vision sensors or inertial sensors are used individually. Considering that each sensor modality has its own limitations, in a number of previously published papers, it has been shown that the fusion of vision and inertial sensor data improves the accuracy of recognition. This survey article provides an overview of the recent investigations where both vision and inertial sensors are used together and simultaneously to perform human action recognition more effectively. The thrust of this survey is on the utilization of depth cameras and inertial sensors as these two types of sensors are cost-effective, commercially available, and more significantly they both provide 3D human action data. An overview of the components necessary to achieve fusion of data from depth and inertial sensors is provided. In addition, a review of the publicly available datasets that include depth and inertial data which are simultaneously captured via depth and inertial sensors is presented.