Deep Imbalanced Learning for Face Recognition and Attribute Prediction

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 42; no. 11; pp. 2781 - 2794
• Main Authors: Huang, Chen, Li, Yining, Loy, Chen Change, Tang, Xiaoou
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; attribute prediction; Clusters; deep convolutional neural networks; Face; Face recognition; Hyperspheres; Image segmentation; Imbalanced learning; Machine learning; Protocols; Representations; Semantics; Skewed distributions; Task analysis; Training
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2019.2914680

Data for face analysis often exhibit highly-skewed class distribution, i.e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances. To mitigate this issue, contemporary deep learning methods typically follow classic strategies such as class re-sampling or cost-sensitive training. In this paper, we conduct extensive and systematic experiments to validate the effectiveness of these classic schemes for representation learning on class-imbalanced data. We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain inter-cluster margins both within and between classes. This tight constraint effectively reduces the class imbalance inherent in the local data neighborhood, thus carving much more balanced class boundaries locally. We show that it is easy to deploy angular margins between the cluster distributions on a hypersphere manifold. Such learned Cluster-based Large Margin Local Embedding (CLMLE), when combined with a simple k-nearest cluster algorithm, shows significant improvements in accuracy over existing methods on both face recognition and face attribute prediction tasks that exhibit imbalanced class distribution.