Wearable Finger Tracking and Cutaneous Haptic Interface with Soft Sensors for Multi-Fingered Virtual Manipulation

• Published in: IEEE/ASME transactions on mechatronics Vol. 24; no. 1; pp. 67 - 77
• Main Authors: Lee, Yongjun, Kim, Myungsin, Lee, Yongseok, Kwon, Junghan, Park, Yong-Lae, Lee, Dongjun
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Contact force; Cutaneous haptic feedback; Degrees of freedom; Electromagnetic interference; Feedback; finger motion tracking; Force; Haptic interfaces; Human performance; Human-computer interaction; Immersive virtual reality; Inertial platforms; Inertial sensing devices; Magnetic sensors; Modules; Sensors; soft sensors; Thumb; Tracking; Virtual environments; Virtual reality; virtual reality (VR); wearable haptic interface; Wearable sensors; Wearable technology
• ISSN: 1083-4435; 1941-014X
• DOI: 10.1109/TMECH.2018.2872570

Multi-Fingered haptics is imperative for truly immersive virtual reality experience, as many real-world tasks involve finger manipulation. One of the key lacking aspect for this is the absence of technologically and economically viable wearable haptic interfaces that can simultaneously track the finger/hand motions and display multi-degree-of-freedom (DOF) contact forces. In this paper, we propose a novel wearable cutaneous haptic interface (WCHI), which consists of 1) finger tracking modules (FTMs) to estimate complex multi-DOF finger and hand motion; and 2) cutaneous haptic modules (CHMs) to convey three-DOF contact force at the finger-tip. By opportunistically utilizing such different types of sensors as inertial measurement units, force sensitive resistor sensors, and soft sensors, the WCHI can track complex anatomically consistent multi-DOF finger motion while avoiding FTM-CHM electromagnetic interference possibly stemming from their collocation in the small form-factor interface; while also providing the direction and magnitude of three-DOF finger-tip contact force, the feedback of which can significantly enhance the precision of contact force generation against variability among users via their closed-loop control. Human subject study is performed with a virtual peg insertion task to show the importance of both the multi-DOF finger tracking and the three-DOF cutaneous haptic feedback for dexterous manipulation in virtual environment.