Face Association for Videos Using Conditional Random Fields and Max-Margin Markov Networks

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 38; no. 9; pp. 1762 - 1773
• Main Authors: Ming Du, Chellappa, Rama
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.09.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; conditional random field; contextual features; Face; Face association; Feature extraction; Hidden Markov models; Markov random fields; max-margin Markov networks; Target tracking; tracking by detection; Videos
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2015.2497689

We address the video-based face association problem, in which one attempts to extract the face tracks of multiple subjects while maintaining label consistency. Traditional tracking algorithms have difficulty in handling this task, especially when challenging nuisance factors like motion blur, low resolution or significant camera motions are present. We demonstrate that contextual features, in addition to face appearance itself, play an important role in this case. We propose principled methods to combine multiple features using Conditional Random Fields and Max-Margin Markov networks to infer labels for the detected faces. Different from many existing approaches, our algorithms work in online mode and hence have a wider range of applications. We address issues such as parameter learning, inference and handling false positves/negatives that arise in the proposed approach. Finally, we evaluate our approach on several public databases.