Tactile Facial Action Units Toward Enriching Social Interactions for Individuals Who Are Blind

• Published in: Smart Multimedia pp. 3 - 14
• Main Authors: McDaniel, Troy, Devkota, Samjhana, Tadayon, Ramin, Duarte, Bryan, Fakhri, Bijan, Panchanathan, Sethuraman
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Assistive technology; Facial action units; Sensory substitution; Social assistive aids; Visual-to-tactile mapping
• ISBN: 9783030043742; 3030043746
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-04375-9_1

Social interactions mediate our communication with others, enable development and maintenance of personal and professional relationships, and contribute greatly to our health. While both verbal cues (i.e., speech) and non-verbal cues (e.g., facial expressions, hand gestures, and body language) are exchanged during social interactions, the latter encompasses more information (~65%). Given their inherent visual nature, non-verbal cues are largely inaccessible to individuals who are blind, putting this population at a social disadvantage compared to their sighted peers. For individuals who are blind, embarrassing social situations are not uncommon due to miscommunication, which can lead to social avoidance and isolation. In this paper, we propose a mapping between visual facial expressions, represented as facial action units, which may be extracted using computer vision algorithms, to haptic (vibrotactile) representations, toward discreet and real-time perception of facial expressions during social interactions by individuals who are blind.