Human Mesh Reconstruction with Generative Adversarial Networks from Single RGB Images

• Published in: Sensors (Basel, Switzerland) Vol. 21; no. 4; p. 1350
• Main Authors: Gao, Rui, Wen, Mingyun, Park, Jisun, Cho, Kyungeun
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI 14.02.2021; MDPI AG
• Subjects: 3D human model; artificial intelligence; Cities; deep learning; GAN; Human Body; Humans; image processing; Image Processing, Computer-Assisted; Neural Networks, Computer; smart cities; User-Computer Interface; Cities; image processing; smart cities; deep learning; 3D human model; GAN; artificial intelligence
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21041350

Applications related to smart cities require virtual cities in the experimental development stage. To build a virtual city that are close to a real city, a large number of various types of human models need to be created. To reduce the cost of acquiring models, this paper proposes a method to reconstruct 3D human meshes from single images captured using a normal camera. It presents a method for reconstructing the complete mesh of the human body from a single RGB image and a generative adversarial network consisting of a newly designed shape-pose-based generator (based on deep convolutional neural networks) and an enhanced multi-source discriminator. Using a machine learning approach, the reliance on multiple sensors is reduced and 3D human meshes can be recovered using a single camera, thereby reducing the cost of building smart cities. The proposed method achieves an accuracy of 92.1% in body shape recovery; it can also process 34 images per second. The method proposed in this paper approach significantly improves the performance compared with previous state-of-the-art approaches. Given a single view image of various humans, our results can be used to generate various 3D human models, which can facilitate 3D human modeling work to simulate virtual cities. Since our method can also restore the poses of the humans in the image, it is possible to create various human poses by given corresponding images with specific human poses.