Real-time facial landmark detection

• Main Authors: Goldin, Ishay, Levy, Noga, Shapira, Gil, Jevnisek, Roy
• Format: Patent
• Language: English
• Published: 07.02.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

Embodiments of the present disclosure enable and accurate detection of facial landmarks on mobile devices in real-time. An architecture of a facial landmark detection model is provided including one or more of an attention mechanism (e.g., an attention network), a graph convolution model (e.g., a two-dimensional facial geometry graph convolution model), a multiscale coarse-to-fine mechanism, a patch-facial landmark detachment mechanism, and error estimation techniques. The attention mechanism may increase the accuracy of the facial landmark detection model by attending to meaningful patches. The graph convolution network may improve patch feature aggregation by considering the facial landmarks' geometry. The coarse-to-fine mechanism reduces a network convergence to two cycles (e.g., two facial landmark detection iterations). A patch-facial landmark detachment mechanism reduces the computation burden without significant accuracy degradation. Error estimation techniques provide accurate estimation of the regression error to the computation load and increase the accuracy of the model.