Natural Eye Motion Synthesis by Modeling Gaze-Head Coupling

• Published in: 2009 IEEE Virtual Reality Conference pp. 143 - 150
• Main Authors: Ma, Xiaohan, Deng, Zhigang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2009
• Subjects: Avatars; Cameras; Computer graphics; Coupled mode analysis; Data acquisition; Digital Avatars; Eye Motion; Facial animation; Gaze-Head Coupling; H.5.2 [Information Interfaces and Presentation]: User Interfaces-Graphical user interfaces (GUI); Humans; I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism-Animation; Magnetic heads; Optical recording; Virtual reality
• ISBN: 9781424439430; 1424439434
• ISSN: 1087-8270
• DOI: 10.1109/VR.2009.4811014

Due to the intrinsic subtlety and dynamics of eye movements, automated generation of natural and engaging eye motion has been a challenging task for decades. In this paper we present an effective technique to synthesize natural eye gazes given a head motion sequence as input, by statistically modeling the innate coupling between gazes and head movements. We first simultaneously recorded head motions and eye gazes of human subjects, using a novel hybrid data acquisition solution consisting of an optical motion capture system and off-the-shelf video cameras. Then, we statistically learn gaze-head coupling patterns using a dynamic coupled component analysis model. Finally, given a head motion sequence as input, we can synthesize its corresponding natural eye gazes based on the constructed gaze-head coupling model. Through comparative user studies and evaluations, we found that comparing with the state of the art algorithms in eye motion synthesis, our approach is more effective to generate natural gazes correlated with given head motions. We also showed the effectiveness of our approach for gaze simulation in two-party conversations.