Selective Transfer Machine for Personalized Facial Expression Analysis

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 39; no. 3; pp. 529 - 545
• Main Authors: Chu, Wen-Sheng, De la Torre, Fernando, Cohn, Jeffrey F.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classifiers; domain adaptation; Face; Facial expression analysis; Feature extraction; Gold; Hidden Markov models; Learning; personalization; Shape; support vector machine (SVM); Teaching methods; Training; Training data; transfer learning
• ISSN: 0162-8828; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2016.2547397

Automatic facial action unit (AU) and expression detection from videos is a long-standing problem. The problem is challenging in part because classifiers must generalize to previously unknown subjects that differ markedly in behavior and facial morphology (e.g., heavy versus delicate brows, smooth versus deeply etched wrinkles) from those on which the classifiers are trained. While some progress has been achieved through improvements in choices of features and classifiers, the challenge occasioned by individual differences among people remains. Person-specific classifiers would be a possible solution but for a paucity of training data. Sufficient training data for person-specific classifiers typically is unavailable. This paper addresses the problem of how to personalize a generic classifier without additional labels from the test subject. We propose a transductive learning method, which we refer to as a Selective Transfer Machine (STM), to personalize a generic classifier by attenuating person-specific mismatches. STM achieves this effect by simultaneously learning a classifier and re-weighting the training samples that are most relevant to the test subject. We compared STM to both generic classifiers and cross-domain learning methods on four benchmarks: CK+ [44], GEMEP-FERA [67], RUFACS [4] and GFT [57]. STM outperformed generic classifiers in all.