Two-stream Prototype Learning Network for Few-shot Face Recognition under Occlusions

• Published in: IEEE transactions on multimedia Vol. 25; pp. 1 - 9
• Main Authors: Yang, Xingyu, Han, Mengya, Luo, Yong, Hu, Han, Wen, Yonggang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Face recognition; Face recognition under occlusions; feature alignment; Feature extraction; few-shot learning; Image quality; Image recognition; Learning; Occlusion; Prototypes; Queries; similarity; Streaming media; Task analysis; Training; two-stream
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2023.3253054

Few-shot face recognition under occlusion (FSFRO) aims to recognize novel subjects given only a few, probably occluded face images, and it is challenging and common in real-world scenarios. Unknown occlusions may deteriorate the class prototypes, while an occluded image in the support set may be critical for recognition if the query image is occluded. This motivates us to propose a novel Two-stream Prototype Learning Network (TSPLN) for FSFR under occlusions by simultaneously considering the quality of support images and their relevance to the query i mage. Specifically, we design a two-stream architecture, which mainly consists of a support-centered stream and query-centered stream, to learn the optimal class prototypes. The former stream is to reduce the negative impact of occluded images on the prototype. This is achieved by exploring the similarities between different images in the support set. In the query-centered stream, we exploit the relevance between the query and support set based on feature alignment (FA). We conduct extensive experiments on two popular datasets: CASIA-WebFace and RMFRD. The experimental results show that our proposed method achieves the state-of-the-art performance for occluded face recognition in the few-shot setting.