Pose2UV: Single-Shot Multiperson Mesh Recovery With Deep UV Prior

• Published in: IEEE transactions on image processing Vol. 31; pp. 4679 - 4692
• Main Authors: Huang, Buzhen, Zhang, Tianshu, Wang, Yangang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Benchmark testing; Color; Color imagery; Detectors; Feature extraction; Heat recovery; human prior; Image reconstruction; Multi-person mesh recovery; Recovery; Shape; Three-dimensional displays; visible pose and mask estimation; Websites
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2022.3187294

In this work, we focus on the task of multi-person mesh recovery from a single color image, where the key issue is to tackle the pixel-level ambiguities caused by inter-person occlusions. Overall, there are two main technical challenges when addressing the ambiguities: how to extract valid target features under occlusions and how to reconstruct reasonable human meshes with only a handful of body cues? To deal with these problems, our key idea is to utilize the predicted 2D poses to locate and separate the target person, and reconstruct them with a novel learning-based UV prior. Specifically, we propose a visible pose-mask module to help extract valid target features, then train a dense body mesh prior to promote reconstructing natural mesh represented by the UV position map. To evaluate the performance of our proposed method under occlusions, we further build an in-the-wild 3D multi-person benchmark named as 3DMPB. Experimental results demonstrate that our method achieves state-of-the-art compared with previous methods. The dataset, codes are publicly available on our website.