Personalized Convolution for Face Recognition

• Published in: International journal of computer vision Vol. 130; no. 2; pp. 344 - 362
• Main Authors: Han, Chunrui, Shan, Shiguang, Kan, Meina, Wu, Shuzhe, Chen, Xilin
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2022; Springer; Springer Nature B.V
• Subjects: Artificial Intelligence; Artificial neural networks; Biometry; Commonality; Computer Imaging; Computer Science; Computers; Customization; Face recognition; Feature extraction; Image Processing and Computer Vision; Kernels; Machine learning; Methods; Neural networks; Pattern Recognition; Pattern Recognition and Graphics; Vision; Face recognition; Personalized convolution; Personalized kernel
• DOI: 10.1007/s11263-021-01536-x
• ISSN: 0920-5691

Face recognition has been significantly advanced by deep learning based methods. In all face recognition methods based on convolutional neural network (CNN), the convolutional kernels for feature extraction are fixed regardless of the input face once the training stage is finished. By contrast, we humans are usually impressed by some unique characteristics of different persons, such as one’s blue eyes while another one’s crooked nose, or even someone’s naevus at specific location. Inspired by this observation, we propose a personalized convolution method which aims to extract special distinguishing characteristics of each person for more accurate face recognition. Specifically, given a face, we adaptively generate a set of kernels for him/her, named by us ordinary kernel, which is further analytically decomposed into two orthogonal components, i.e., the commonality component and the specialty component. The former characterizes the commonality among subjects which is optimized on a reference set. The latter is the residual part by filtering out the commonality component from the ordinary kernel, so as to capture those special characteristics, named by us personalized kernel. The CNNs with personalized kernels for convolution can highlight those specialty of a person’s distinguishing characteristics while suppress his/her commonality with others, leading to better distinguishing of different faces. Additionally, as a by-product, the reference set also facilitates the adaptation of our method to different scenarios by simply selecting faces of a particular population. Extensive experiments on the challenging LFW, IJB-A and IJB-C datasets validate that our proposed personalized convolution achieves significant improvement over the conventional CNN, and also other existing methods for face recognition.