Content-Aware Eye Tracking for Autostereoscopic 3D Display

• Published in: Sensors (Basel, Switzerland) Vol. 20; no. 17; p. 4787
• Main Authors: Kang, Dongwoo, Heo, Jingu
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 25.08.2020; MDPI
• Subjects: Accuracy; Algorithms; augmented reality display; Autostereoscopic; autostereoscopic three-dimensional display; Consumer electronics; content-aware eye alignment; Crosstalk; Digital cameras; Display devices; error reinforcement learning; eye detection; Eye movements; eye tracking; Eye-Tracking Technology; Eyeglasses; Eyewear; Face recognition; Image classification; Imaging, Three-Dimensional; Infrared cameras; Light; Light emitting diodes; Machine learning; Methods; Near infrared radiation; Neural networks; Nose; Smartphones; autostereoscopic three-dimensional display; augmented reality display; error reinforcement learning; eye tracking; content-aware eye alignment; eye detection
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s20174787

This study develops an eye tracking method for autostereoscopic three-dimensional (3D) display systems for use in various environments. The eye tracking-based autostereoscopic 3D display provides low crosstalk and high-resolution 3D image experience seamlessly without 3D eyeglasses by overcoming the viewing position restriction. However, accurate and fast eye position detection and tracking are still challenging, owing to the various light conditions, camera control, thick eyeglasses, eyeglass sunlight reflection, and limited system resources. This study presents a robust, automated algorithm and relevant systems for accurate and fast detection and tracking of eye pupil centers in 3D with a single visual camera and near-infrared (NIR) light emitting diodes (LEDs). Our proposed eye tracker consists of eye-nose detection, eye-nose shape keypoint alignment, a tracker checker, and tracking with NIR LED on/off control. Eye-nose detection generates facial subregion boxes, including the eyes and nose, which utilize an Error-Based Learning (EBL) method for the selection of the best learnt database (DB). After detection, the eye-nose shape alignment is processed by the Supervised Descent Method (SDM) with Scale-invariant Feature Transform (SIFT). The aligner is content-aware in the sense that corresponding designated aligners are applied based on image content classification, such as the various light conditions and wearing eyeglasses. The conducted experiments on real image DBs yield promising eye detection and tracking outcomes, even in the presence of challenging conditions.