Learning Race from Face: A Survey

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 36; no. 12; pp. 2483 - 2509
• Main Authors: Fu, Siyao, He, Haibo, Hou, Zeng-Guang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2014; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Anthropometry - methods; Biometric Identification - methods; Categories; Computational modeling; Computer simulation; Computer vision; Continental Population Groups - classification; Cultural differences; Face - anatomy & histology; Face recognition; Feature extraction; Female; Humans; Image classification; Image color analysis; Image Processing, Computer-Assisted - methods; Intelligence; Learning; Machine Learning; Male; Mathematical models; Perception; Psychology; Race; computer vision; face recognition; data clustering; Race classification; face database; image categorization; machine learning
• DOI: 10.1109/TPAMI.2014.2321570
• ISSN: 0162-8828

Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.