Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People

• Published in: IEEE transactions on mobile computing Vol. 18; no. 3; pp. 702 - 714
• Main Authors: Li, Bing, Munoz, Juan Pablo, Rong, Xuejian, Chen, Qingtian, Xiao, Jizhong, Tian, Yingli, Arditi, Aries, Yousuf, Mohammed
• Format: Magazine Article
• Language: English
• Published: United States IEEE 01.03.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Ambient intelligence; blind and visually impaired people; Blind people; Blindness; Cameras; Collision avoidance; Google; Google Tango device; Indoor assistive navigation; Kalman filters; Man-machine interfaces; Navigation; Navigation safety; Navigation systems; Navigational aids; Obstacle avoidance; Path planning; People with disabilities; Real-time systems; semantic maps; Semantics; Two dimensional models; Wayfinding; blind and visually impaired people; Indoor assistive navigation; obstacle avoidance; semantic maps; Google Tango device
• ISSN: 1536-1233; 1558-0660; 1558-0660
• DOI: 10.1109/TMC.2018.2842751

This paper presents a new holistic vision-based mobile assistive navigation system to help blind and visually impaired people with indoor independent travel. The system detects dynamic obstacles and adjusts path planning in real-time to improve navigation safety. First, we develop an indoor map editor to parse geometric information from architectural models and generate a semantic map consisting of a global 2D traversable grid map layer and context-aware layers. By leveraging the visual positioning service (VPS) within the Google Tango device, we design a map alignment algorithm to bridge the visual area description file (ADF) and semantic map to achieve semantic localization. Using the on-board RGB-D camera, we develop an efficient obstacle detection and avoidance approach based on a time-stamped map Kalman filter (TSM-KF) algorithm. A multi-modal human-machine interface (HMI) is designed with speech-audio interaction and robust haptic interaction through an electronic SmartCane. Finally, field experiments by blindfolded and blind subjects demonstrate that the proposed system provides an effective tool to help blind individuals with indoor navigation and wayfinding.