Sign Language Recognition Based on Trajectory Modeling with HMMs

• Published in: MultiMedia Modeling pp. 686 - 697
• Main Authors: Pu, Junfu, Zhou, Wengang, Zhang, Jihai, Li, Houqiang
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Curve Feature; Gesture Recognition; Hide Markov Model; Shape Context; Sign Language
• ISBN: 9783319276700; 3319276700
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-27671-7_58

Sign language recognition targets on interpreting and understanding the sign language for convenience of communication between the deaf and the normal people, which has broad social impact. The problem is challenging due to the large variations for different signers and the subtle difference between sign words. In this paper, we propose a new method for isolated sign language recognition based on trajectory modeling with hidden Markov models (HMMs). In our approach, we first normalize and re-sample the raw trajectory data and partition the trajectory into multiple segments. To represent each trajectory segment, we proposed a new curve feature descriptor based on shape context. After that, hidden Markov model is used to model each isolated sign word for recognition. To evaluate the performance of our proposed algorithm, we have built a large isolated Chinese sign language vocabulary with Kinect 2.0. The dataset contains 100 unique isolated sign words, each of which is performed by 50 signers for 5 times. Experimental results demonstrate that the proposed method achieves a better performance compared with normal coordinate feature with HMM.