Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model

• Published in: Multimedia tools and applications Vol. 80; no. 14; pp. 21465 - 21498
• Main Authors: Nadeem, Amir, Jalal, Ahmad, Kim, Kibum
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2021; Springer Nature B.V
• Subjects: Activity recognition; Body parts; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Discriminant analysis; Emission analysis; Entropy; Fitness; Markov chains; Maximum entropy; Multimedia Information Systems; Optical flow (image analysis); Robustness; Sensors; Service robots; Special Purpose and Application-Based Systems; Statistical analysis; Statistical methods; Virtual reality; Entropy Markov model; Multidimensional cues; Body parts detection; Sports activity recognition; Posture estimation
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-021-10687-5

Automated human posture estimation (A-HPE) systems need delicate methods for detecting body parts and selecting cues based on marker-less sensors to effectively recognize complex activity motions. Recognition of human activities using vision sensors is a challenging issue due to variations in illumination conditions and complex movements during the monitoring of sports and fitness exercises. In this paper, we propose a novel A-HPE method that intelligently identifies human behaviours by utilizing saliency silhouette detection, robust body parts model and multidimensional cues from full-body silhouettes followed by an entropy Markov model. Initially, images are pre-processed and noise is removed to obtain a robust silhouette. Body parts models are then used to extract twelve key body parts. These key body parts are further optimized to assist the generation of multidimensional cues. These cues include energy, optical flow and distinctive values that are fed into quadratic discriminant analysis to discriminate cues which help in the recognition of actions. Finally, these optimized patterns are further processed by a maximum entropy Markov model as a recognizer engine based on transition and emission probability values for activity recognition. For evaluation, we used a leave-one-out cross validation scheme and the results outperformed existing well-known statistical state-of-the-art methods by achieving better body parts detection and higher recognition accuracy over four benchmark datasets. The proposed method will be useful for man-machine interactions such as 3D interactive games, virtual reality, service robots, e-health fitness, and security surveillance. Graphical Abstract Design model of automatic posture estimation and action recognition.