Generating photo-realistic training data to improve face recognition accuracy

• Published in: Neural networks Vol. 134; pp. 86 - 94
• Main Authors: Sáez Trigueros, Daniel, Meng, Li, Hartnett, Margaret
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.02.2021
• Subjects: Automated Facial Recognition - methods; Face and gesture recognition; Facial Recognition - physiology; Generative adversarial learning; Humans; Image generation; Machine Learning; Neural Networks, Computer; Photography - methods; Generative adversarial learning; Face and gesture recognition; Image generation; Machine learning
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2020.11.008

Face recognition has become a widely adopted biometric in forensics, security and law enforcement thanks to the high accuracy achieved by systems based on convolutional neural networks (CNNs). However, to achieve good performance, CNNs need to be trained with very large datasets which are not always available. In this paper we investigate the feasibility of using synthetic data to augment face datasets. In particular, we propose a novel generative adversarial network (GAN) that can disentangle identity-related attributes from non-identity-related attributes. This is done by training an embedding network that maps discrete identity labels to an identity latent space that follows a simple prior distribution, and training a GAN conditioned on samples from that distribution. A main novelty of our approach is the ability to generate both synthetic images of subjects in the training set and synthetic images of new subjects not in the training set, both of which we use to augment face datasets. By using recent advances in GAN training, we show that the synthetic images generated by our model are photo-realistic, and that training with datasets augmented with those images can lead to increased recognition accuracy. Experimental results show that our method is more effective when augmenting small datasets. In particular, an absolute accuracy improvement of 8.42% was achieved when augmenting a dataset of less than 60k facial images. •Generate photo-realistic face images using a conditional GAN.•Two latent vectors encode identity-related and non-related attributes respectively.•Map discrete identity labels to identity features in a continuous latent space.•Training sets with better balance between real and synthetic images outperformed.