Gesture control interface for immersive panoramic displays

• Published in: Multimedia tools and applications Vol. 73; no. 1; pp. 491 - 517
• Main Authors: Alcoverro, Marcel, Suau, Xavier, Morros, Josep R., López-Méndez, Adolfo, Gil, Albert, Ruiz-Hidalgo, Javier, Casas, Josep R.
• Format: Journal Article Publication
• Language: English
• Published: Boston Springer US 01.11.2014; Springer; Springer Nature B.V
• Subjects: Analysis; Applied sciences; Artificial intelligence; Automation; Computer Communication Networks; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Control de processos; Control systems; Data Structures and Information Theory; Displays; Exact sciences and technology; Face recognition; Human factors; Human-machine systems; Illumination; Informàtica; Interacció home-màquina; Interacció persona-ordinador; Interfaces; Localization; Multimedia; Multimedia Information Systems; Nonverbal communication; Pattern recognition systems; Pattern recognition. Digital image processing. Computational geometry; Process control; Proposals; Recognition; Reconeixement facial (Informàtica); Sistemes d'informació; Sistemes persona-màquina; Software; Special Purpose and Application-Based Systems; State of the art; Studies; Systems design; Television sets; Tracking; Usability; User interface; User interfaces (Computer systems); Voice recognition; Àrees temàtiques de la UPC; Interactivity; Panoramic display; Human-machine interfaces; Computer vision; Motion estimation; Image processing; Face recognition; Hand; Omnidirectional vision; Luminance; Experimental result; User interface; Authentication; Facies; Gesture; Virtual reality; Illumination; Usability
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-013-1605-7

In this paper, we propose a gesture-based interface designed to interact with panoramic scenes. The system combines novel static gestures with a fast hand tracking method. Our proposal is to use static gestures as shortcuts to activate functionalities of the system (i.e. volume up/down, mute, pause, etc.), and hand tracking to freely explore the panoramic video. The overall system is multi-user, and incorporates a user identification module based on face recognition, which is able both to recognize returning users and to add new users online. The system exploits depth data, making it robust to challenging illumination conditions. We show through experimental results the performance of every component of the system compared to the state of the art. We also show the results of a usability study performed with several untrained users.