Microwave Reconstruction of 3-D Human Facial Landmarks Using a Programmable Metasurface

• Published in: IEEE antennas and wireless propagation letters Vol. 23; no. 11; pp. 3327 - 3331
• Main Authors: Li, Mingyi, Xu, Jiawen, Cui, Tie Jun, Li, Lianlin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Body parts; Deep learning; Encoding; Faces; facial reconstruction (FR); Frequency ranges; Image reconstruction; Metasurfaces; Microwave imaging; Microwave theory and techniques; Occlusion; Optical measuring instruments; programmable metasurface; Three-dimensional displays; Virtual reality
• ISSN: 1536-1225; 1548-5757
• DOI: 10.1109/LAWP.2024.3403686

Nowadays, 3-D human facial reconstruction (FR) plays a pivotal role in various areas including human-robot interaction, virtual reality, identification, and so on. However, conventional FR strategies are largely based on optical sensors, and thus suffer from the intrinsic risk in terms of obstacle (e.g., mask) occlusion, light-condition dependence, privacy infringement, and so on. To tackle it, we here propose to leverage the microwave for the 3-D FR, by utilizing a fundamental fact that the microwave, unlike optics, is capable of operating in the conditions of all-weather, all-day, and occlusion. To this end, we design an inexpensive transceiver architecture based on a large-aperture programmable metasurface, enabling to capture fully the 3-D facial information of the subject, meanwhile suppressing the unwanted disturbances from surrounding environment and other body parts. Afterward, we design a deep artificial network for converting the microwave raw data into the 3-D coordinates of 68 facial landmarks. We fabricate a proof-of-principle prototype system working within the frequency range of (25 to 26.5) GHz, and experimentally demonstrate that the reported strategy is efficient in the reconstruction of 3-D human facial landmarks even for the subject wearing a mask.