Learning feature alignment across attribute domains for improving facial beauty prediction

• Published in: Expert systems with applications Vol. 249; p. 123644
• Main Authors: Sun, Zhishu, Lin, Luojun, Yu, Yuanlong, Jin, Lianwen
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.09.2024
• Subjects: Batch normalization; Dynamic convolution; Facial attractiveness assessment; Facial beauty prediction; Dynamic convolution; Facial attractiveness assessment; Facial beauty prediction; Batch normalization
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2024.123644

Facial beauty prediction (FBP) aims to develop a system to assess facial attractiveness automatically. Through prior research and our own observations, it has become evident that attribute information, such as gender and race, is a key factor leading to the distribution discrepancy in the FBP data. Such distribution discrepancy hinders current conventional FBP models from generalizing effectively to unseen attribute domain data, thereby discounting further performance improvement. To address this problem, in this paper, we exploit the attribute information to guide the training of convolutional neural networks (CNNs), with the final purpose of implicit feature alignment across various attribute domain data. To this end, we introduce the attribute information into convolution layer and batch normalization (BN) layer, respectively, as they are the most crucial parts for representation learning in CNNs. Specifically, our method includes: 1) Attribute-guided convolution (AgConv) that dynamically updates convolutional filters based on attributes by parameter tuning or parameter rebirth; 2) Attribute-guided batch normalization (AgBN) is developed to compute the attribute-specific statistics through an attribute guided batch sampling strategy; 3) To benefit from both approaches, we construct an integrated framework by combining AgConv and AgBN to achieve a more thorough feature alignment across different attribute domains. Extensive qualitative and quantitative experiments have been conducted on the SCUT-FBP, SCUT-FBP5500 and HotOrNot benchmark datasets. The results show that AgConv significantly improves the attribute-guided representation learning capacity and AgBN provides more stable optimization. Owing to the combination of AgConv and AgBN, the proposed framework (Ag-Net) achieves further performance improvement and is superior to other state-of-the-art approaches for FBP. •Deep learning techniques for facial beauty prediction.•Learning feature alignment across multiple attribute domain data.•Learning attribute-guided feature representation for better feature alignment.•Attribute-guided convolution to dynamically adjusts convolutional parameters.•Attribute-guided batch normalization to compute attribute-specific statistics.