The THU multi-view face database for videoconferences and baseline evaluations

• Published in: Neurocomputing (Amsterdam) Vol. 207; pp. 48 - 59
• Main Authors: Tao, Xiaoming, Dong, Linhao, Li, Yang, Lu, Jianhua
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 26.09.2016
• Subjects: Evaluation protocol; Face video database; Gaze-correction; Multi-view; Videoconferences; Evaluation protocol; Multi-view; Face video database; Gaze-correction; Videoconferences
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2016.03.055

In this paper, we present a face video database and its acquisition. This database contains 31,500 video clips of 100 individual figures from 20 countries. The primary purpose of building this database is to serve as a standardized test video sequences for any research related to videoconferences, such as gaze-correction and model based face reconstruction, etc. To be specific, each of the subjects was filmed by 9 groups of synchronized webcams under 7 illumination conditions, and was requested to complete a series of designated actions. Thus, face variations including lip shape, occlusion, illumination, pose, and expression are well presented in each video clip. Compared to the existing databases, the proposed THU face database provides multi-view video sequences with strict temporal synchronization, which enables evaluations on current and future possible gaze-correction methods for conversational video communications. Besides, we discuss the evaluation protocol based on our database for gaze-correction, where three well-known methods were tested. Experiment results show that, under this evaluation protocol, comparisons of performance can be obtained numerically in terms of peak signal-to-noise ratio (PSNR), demonstrating the strengths and weaknesses of these methods under different circumstances.