Real-time human body tracking based on data fusion from multiple RGB-D sensors

• Published in: Multimedia tools and applications Vol. 76; no. 3; pp. 4249 - 4271
• Main Authors: Núñez, Juan C., Cabido, Raúl, Montemayor, Antonio S., Pantrigo, Juan J.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2017; Springer Nature B.V
• Subjects: Algorithms; Cameras; Computer Communication Networks; Computer Science; Computer vision; Data integration; Data Structures and Information Theory; Devices; Frames per second; Human body; Methods; Movement; Multimedia; Multimedia communications; Multimedia Information Systems; Position tracking; Sensors; Smoothness; Special Purpose and Application-Based Systems; Studies; Tracking; United States > US; RGBD sensors; Sensor fusion; Human body tracking
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-016-3759-6

In this work we present a human pose estimation method based on the skeleton fusion and tracking using multiple RGB-D sensors. The proposed method considers the skeletons provided by each RGB-D device and constructs an improved skeleton, taking into account the quality measures provided by the sensors at two different levels: the whole skeleton and each joint individually. Then, each joint is tracked by a Kalman filter, resulting in a smooth tracking performance. We have also developed a new dataset consisting of six subjects performing seven different gestures, recorded with four Kinect devices simultaneously. Experimental results performed on this dataset show that the system obtains better smoothness results than the most representative methods found in the literature. The proposed system operates at a processing rate of 25 frames per second (including the whole algorithm loop, i.e., data acquisition and processing) without the explicit use of the multithreading capabilities of the system.