Adaptively Weighted k-Tuple Metric Network for Kinship Verification

• Published in: IEEE transactions on cybernetics Vol. 53; no. 10; pp. 6173 - 6186
• Main Authors: Huang, Sheng, Lin, Jingkai, Huangfu, Luwen, Xing, Yun, Hu, Junlin, Zeng, Daniel Dajun
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computer vision; Convolutional neural networks; Deep learning; Faces; Feature extraction; Genetics; Human performance; kinship verification; Measurement; metric learning; Performance enhancement; relation network (RN); Task analysis; triplet loss; Verification
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2022.3163707

Facial image-based kinship verification is a rapidly growing field in computer vision and biometrics. The key to determining whether a pair of facial images has a kin relation is to train a model that can enlarge the margin between the faces that have no kin relation while reducing the distance between faces that have a kin relation. Most existing approaches primarily exploit duplet (i.e., two input samples without cross pair) or triplet (i.e., single negative pair for each positive pair with low-order cross pair) information, omitting discriminative features from multiple negative pairs. These approaches suffer from weak generalizability, resulting in unsatisfactory performance. Inspired by human visual systems that incorporate both low-order and high-order cross-pair information from local and global perspectives, we propose to leverage high-order cross-pair features and develop a novel end-to-end deep learning model called the adaptively weighted <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-tuple metric network (AW<inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-TMN). Our main contributions are three-fold. First, a novel cross-pair metric learning loss based on <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-tuplet loss is introduced. It naturally captures both the low-order and high-order discriminative features from multiple negative pairs. Second, an adaptively weighted scheme is formulated to better highlight hard negative examples among multiple negative pairs, leading to enhanced performance. Third, the model utilizes multiple levels of convolutional features and jointly optimizes feature and metric learning to further exploit the low-order and high-order representational power. Extensive experimental results on three popular kinship verification datasets demonstrate the effectiveness of our proposed AW<inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-TMN approach compared with several state-of-the-art approaches. The source codes and models are released.<xref ref-type="fn" rid="fn1">1