AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 45; no. 6; pp. 7157 - 7173
• Main Authors: Fang, Hao-Shu, Li, Jiefeng, Tang, Hongyang, Xu, Chao, Zhu, Haoyi, Xiu, Yuliang, Li, Yong-Lu, Lu, Cewu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computer vision; Datasets; Detectors; Distillation; Faces; hand pose estimation; Heating systems; Human pose estimation; Humans; Location awareness; multi-person; Pose estimation; pose tracking; Posture; Proposals; Real time; realtime; Tracking; Training; whole-body pose estimation
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2022.3222784

Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this article, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose .