Real-time low-cost human skeleton detection

• Published in: Multimedia tools and applications Vol. 80; no. 26-27; pp. 34389 - 34402
• Main Authors: Song, Eungyeol, Do, Jinkyung, Yu, Sunjin
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2021; Springer Nature B.V
• Subjects: 1135T: Social Multimedia Processing; Accuracy; Algorithms; Artificial intelligence; Artificial neural networks; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Feature extraction; Human behavior; Low cost; Machine learning; Methods; Multimedia; Multimedia Information Systems; Neural networks; R&D; Real time; Research & development; Shape recognition; Special Purpose and Application-Based Systems; Skeleton detection; Multi-stage CNN; Human skeleton; CNNs
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-021-11308-x

The human skeleton or deep learning framework is useful for accurately recognizing human behavior and analyzing that behavior across different situations. This work introduces a low-cost human skeleton detection network for detecting human skeleton shapes in real time. The proposed network is divided into two parts: pattern extraction and multi-stage convolutional neural networks (CNNs). In the multi-stage CNN step, we use repeated stages, including two branches for the estimation of the heatmap and the part affinity fields (PAFs). In addition, the network consists of inverted bottleneck layers and separable convolutions to extract features efficiently. With test videos, our method achieved an average video analysis speed of approximately 10.45 fps, which is significantly higher than the value of 4.33 fps achieved by the OpenPose algorithm.