3D Hand Gesture Coding for Sign Language Learning

Hand gesture and sign language are important in facilitating non-verbal communication between human beings. This paper proposes a new approach to code 3d hand gestures for presentation and recognition. The coding is based on 3D hand and finger structure extracted from Leap Motion Controller (LMC). F...

Full description

Saved in:
Bibliographic Details
Published in2016 International Conference on Virtual Reality and Visualization (ICVRV) pp. 407 - 410
Main Authors Yi Ji, Chunping Liu, Shengrong Gong, Weidong Cheng
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.09.2016
Subjects
Assistive technology
Encoding
Gesture recognition
Hand Gesture Coding
Interactive Learning
Leap Motion
Sign Language
Three-dimensional displays
Thumb
Vocabulary
Online AccessGet full text

Cover

More Information
Summary:Hand gesture and sign language are important in facilitating non-verbal communication between human beings. This paper proposes a new approach to code 3d hand gestures for presentation and recognition. The coding is based on 3D hand and finger structure extracted from Leap Motion Controller (LMC). From the 3D coordinates tracked by device, shapes, positions or distances between finger tips are coded and normalized according to a set of rules. This coding process can be completed manually or semi-automatically. The recognition stage is automatic. In experiments, our system applied automatic coding and recognition on a subset of Chinese sign language, which is a special set of meaningful hand gestures. Integrated in a Unity application, the method can recognize learned gestures quickly and accurately. For unknown gestures, the system also provided interactive learning method to further extend recognizable alphabet signs.
DOI:10.1109/ICVRV.2016.74