Communication Support System of Smart Glasses for the Hearing Impaired

• Published in: Computers Helping People with Special Needs Vol. 10896; pp. 225 - 232
• Main Authors: Watanabe, Daiki, Takeuchi, Yoshinori, Matsumoto, Tetsuya, Kudo, Hiroaki, Ohnishi, Noboru
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2018; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Automatic speech recognition; Hearing impaired; Smart glasses; Sound localization; Voice enhancement
• ISBN: 9783319942766; 331994276X
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-94277-3_37

In this research, we propose a novel system for displaying captions of conversation content on smart glasses. We propose natural communication using smart glasses that is more like a conversation between hearing people than conventional communication methods such as sign language or handwriting. The system translates spoken words into text and displays them on the screen of the smart glasses. It equips four microphones, localizes the direction of the sound source, and distinguishes the angular direction of the sound source. Using signals from four microphones, the system can enhance voices. The voice enhancement technique is required to improve the automatic speech recognition rate in noisy environments. Experimental results showed that the system can estimate the angular direction of a voice and recognize more than 90% of words that are spoken. The subject experiment showed that the proposed system had a similarity score close to an existing smartphone application.