Identification and Reconstruction of Human Faces into 3D Models Using SSD-Based and Attention Mesh Models in Real-Time

• Published in: SN computer science Vol. 5; no. 8; p. 1092
• Main Authors: Priyadharshini, A. R., Annamalai, R.
• Format: Journal Article
• Language: English
• Published: Singapore Springer Nature Singapore 01.12.2024; Springer Nature B.V
• Subjects: Access control; Accuracy; Animation; Augmented reality; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Datasets; Efficiency; Facial recognition technology; Image reconstruction; Information Systems and Communication Service; Medical imaging; Methods; Neural networks; Occlusion; Optimization; Original Research; Parameter identification; Pattern Recognition and Graphics; Real time; Software Engineering/Programming and Operating Systems; Virtual reality; Vision; 3DMM; CNN; DFNet; Attention mesh; SSD
• ISSN: 2661-8907; 2662-995X; 2661-8907
• DOI: 10.1007/s42979-024-03481-1

The identification and 3D reconstruction of human faces hold vital significance, influencing augmented reality realism, bolstering biometric security measures, enabling virtual try-on experiences, aiding medical imaging for surgeries, and enhancing animation and entertainment with lifelike character representations. Presently, prevalent approaches enhance computational speed through the utilization of lightweight networks for parameter fitting, albeit sacrificing reconstruction accuracy. We have enhanced the 3D reconstruction model to address these challenges and introduced an efficient and lightweight network architecture. In this paper, we propose a novel pipeline combining facial landmarks, and facial mesh to render 3D Models in real-time dynamically. We combine a Single Shot MultiBox Detector (SSD) to detect faces marking 468 facial landmarks on human faces with a Face attention mesh architecture. The Attention mesh model applies a 3D mesh structure combing the facial landmarks which is then fed into the Babylon js model. Finally, a 3D Model is rendered which dynamically replicates the expression and movements of the subject in real-time video. The experimental outcomes demonstrate that the method outlined in this paper attains high-precision reconstruction with a lightweight framework and exhibits increased resilience to factors like attitude and occlusion.