Spatiotemporal Correlation-Based Accurate 3D Face Imaging Using Speckle Projection and Real-Time Improvement

• Published in: Applied sciences Vol. 11; no. 18; p. 8588
• Main Authors: Xiong, Wei, Yang, Hongyu, Zhou, Pei, Fu, Keren, Zhu, Jiangping
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.09.2021
• Subjects: Accuracy; Algorithms; Computer applications; Correlation; Data acquisition; Experiments; Forecasting; Image acquisition; Infrared imagery; Light; Methods; Microprocessors; Parameters; Protective equipment; Real time; spatiotemporal correlation; Speckle patterns; speckle projection; stereo matching; three-dimensional face reconstruction; Virtual reality
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app11188588

The reconstruction of 3D face data is widely used in the fields of biometric recognition and virtual reality. However, the rapid acquisition of 3D data is plagued by reconstruction accuracy, slow speed, excessive scenes and contemporary reconstruction-technology. To solve this problem, an accurate 3D face-imaging implementation framework based on coarse-to-fine spatiotemporal correlation is designed, improving the spatiotemporal correlation stereo matching process and accelerating the processing using a spatiotemporal box filter. The reliability of the reconstruction parameters is further verified in order to resolve the contention between the measurement accuracy and time cost. A binocular 3D data acquisition device with a rotary speckle projector is used to continuously and synchronously acquire an infrared speckle stereo image sequence for reconstructing an accurate 3D face model. Based on the face mask data obtained by the high-precision industrial 3D scanner, the relationship between the number of projected speckle patterns, the matching window size, the reconstruction accuracy and the time cost is quantitatively analysed. An optimal combination of parameters is used to achieve a balance between reconstruction speed and accuracy. Thus, to overcome the problem of a long acquisition time caused by the switching of the rotary speckle pattern, a compact 3D face acquisition device using a fixed three-speckle projector is designed. Using the optimal combination parameters of the three speckles, the parallel pipeline strategy is adopted in each core processing unit to maximise system resource utilisation and data throughput. The most time-consuming spatiotemporal correlation stereo matching activity was accelerated by the graphical processing unit. The results show that the system achieves real-time image acquisition, as well as 3D face reconstruction, while maintaining acceptable systematic precision.