InstaHMR: Instance-aware One-stage Multi-person Human Mesh Recovery

• Published in: IEEE transactions on visualization and computer graphics Vol. PP; pp. 1 - 14
• Main Authors: Liao, Xinyao, Zhang, Chen, Xu, Jianyao, Su, Wanjuan, Chen, Zhi, Tao, Wenbing
• Format: Journal Article
• Language: English
• Published: United States IEEE 19.04.2024
• Subjects: Detectors; Faces; Feature extraction; Heating systems; Human Mesh Recovery; Human Pose and Shape Estimation; Instance-aware; One-stage; Shape; Task analysis; Three-dimensional displays
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2024.3391764

Human mesh recovery aims to estimate all human meshes within a given image. In this paper, we propose an Instance-aware Multi-person 3D Human Mesh Recovery (InstaHMR) network based on the one-stage framework. Compared to former one-stage methods, instance-aware single person feature is exploited to represent more accurate human mesh. Specifically, we propose the Contextual Instance Guidance (CIG) module which generates instance-aware single person feature by leveraging spatial and channel attention operations. In this way, it preserves more instance-specific information compared to the pixel-level feature used in some existing one-stage methods. Besides, we further introduce two auxiliary losses for better mesh recovery, namely the Human Triplet Planes (HTP) loss and the T-pose Shape (TS) loss. The HTP loss encourages the model to capture subtle differences in human joint positions, while the TS loss facilitates the learning of abstract shape parameters. By incorporating these advancements, our model achieves state-of-the-art results on four multi-person datasets.