Coarse‐To‐Fine 3D Craniofacial Landmark Detection via Heat Kernel Optimization

• Published in: Computer animation and virtual worlds Vol. 36; no. 4
• Main Authors: Xue, Xingfei, Wang, Xuesong, Liu, Weizhou, Wang, Xingce, Zhao, Junli, Wu, Zhongke
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.07.2025; Wiley Subscription Services, Inc
• Subjects: 3D craniofacial landmark detection; Computer animation; Datasets; Flat surfaces; heat kernel; heatmap regression; Localization; Optimization
• ISSN: 1546-4261; 1546-427X
• DOI: 10.1002/cav.70050

ABSTRACT Accurate 3D craniofacial landmark detection is critical for applications in medicine and computer animation, yet remains challenging due to the complex geometry of craniofacial structures. In this work, we propose a coarse‐to‐fine framework for anatomical landmark localization on 3D craniofacial models. First, we introduce a Diffused Two‐Stream Network (DTS‐Net) for heatmap regression, which effectively captures both local and global geometric features by integrating pointwise scalar flow, tangent space vector flow, and spectral features in the Laplace‐Beltrami space. This design enables robust representation of complex anatomical structures. Second, we propose a heat kernel‐based energy optimization method to extract landmark coordinates from the predicted heatmaps. This approach exhibits strong performance across various geometric regions, including boundaries, flat surfaces, and high‐curvature areas, ensuring accurate and consistent localization. Our method achieves state‐of‐the‐art results on both a 3D cranial dataset and the BU‐3DFE facial dataset. The Coarse‐to‐Fine 3D landmark detection framework consists of two sequential stages: (I) heatmap prediction from sampled point clouds using the proposed DTS‐Net, and (II) continuous surface landmark estimation via heat kernel‐driven energy optimization.