Vision-Guided "Point and Click" for Smart Rooms

• Published in: 2007 Second International Conference on Systems and Networks Communications (ICSNC 2007) p. 30
• Main Authors: Adler, M., Reynolds, F.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.08.2007
• Subjects: Focusing; Graphical user interfaces; Humans; Image Processing; Image recognition; Location Awareness; Pervasive computing; Protocols; Service Discovery; Smart cameras; Smart Spaces; Ubiquitous computing; Usability; User interfaces
• ISBN: 0769529380; 9780769529387
• ISSN: 2163-9019
• DOI: 10.1109/ICSNC.2007.84

Pointing is a natural method for humans to use to indicate a focus of their attention. Over the years, the ubiquitous, infra-red remote control has been used to control a variety of appliances by pointing. Despite its success, infra-red based pointing solutions have some limitations. In our research, we have explored other ways for mobile users to select objects inside a smart room via pointing or similar gesture. In this paper we describe the use of mobile phones with cameras as pointing devices to interact with smart environments. Mobile users select an object by simply taking a picture of it. We rely on two key ideas to simplify the image-processing problem: objects that can be recognized visually contain image-processing hints in their descriptions, and all the objects (and their descriptions) in the vicinity of the user can be found by a location-aware discovery protocol. The paper describes the design of a location-aware service discovery protocol, the integration of the service discovery protocol with image capture and analysis and our experience with a smart room application that uses image capture and recognition to select physical objects.