Real-Time Precise Human-Computer Interaction System Based on Gaze Estimation and Tracking

• Published in: Wireless communications and mobile computing Vol. 2021; no. 1
• Main Authors: Huang, Junhao, Zhang, Zhicheng, Xie, Guoping, He, Hui
• Format: Journal Article
• Language: English
• Published: Oxford Hindawi 2021; John Wiley & Sons, Inc
• Subjects: Accuracy; Algorithms; Cameras; Eye movements; Human-computer interaction; Image classification; Keyboards; Methods; Mouse devices; Real time; Sensors; Support vector machines; Tracking; Webcams; Wireless sensor networks
• ISSN: 1530-8669; 1530-8677
• DOI: 10.1155/2021/8213946

Noncontact human-computer interaction has an important value in wireless sensor networks. This work is aimed at achieving accurate interaction on a computer based on auto eye control, using a cheap webcam as the video source. A real-time accurate human-computer interaction system based on eye state recognition, rough gaze estimation, and tracking is proposed. Firstly, binary classification of the eye states (opening or closed) is carried on using the SVM classification algorithm with HOG features of the input eye image. Second, rough appearance-based gaze estimation is implemented based on a simple CNN model. And the head pose is estimated to judge whether the user is facing the screen or not. Based on these recognition results, noncontact mouse control and character input methods are designed and developed to replace the standard mouse and keyboard hardware. Accuracy and speed of the proposed interaction system are evaluated by four subjects. The experimental results show that users can use only a common monocular camera to achieve gaze estimation and tracking and to achieve most functions of real-time precise human-computer interaction on the basis of auto eye control.