Trajectory-based recognition of dynamic Persian sign language using hidden Markov model

• Published in: Computer speech & language Vol. 61; p. 101053
• Main Authors: Azar, Saeideh Ghanbari, Seyedarabi, Hadi
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.05.2020
• Subjects: Classification; Hidden Markov model; Persian sign language; Sign language recognition; Trajectory; Sign language recognition; Hidden Markov model; Trajectory; Persian sign language; Classification
• ISSN: 0885-2308; 1095-8363
• DOI: 10.1016/j.csl.2019.101053

•A dynamic Persian sign language dataset containing 1200 videos of 20 signs performed by 12 individuals is collected.•Hand trajectory and hand shape information is extracted from each frame of the sample videos via a region growing technique.•Hidden Markov model with Gaussian observations is used to model the extracted time-varying trajectories.•The accuracy of 98.13% is obtained and the performance is independent of the subject and the number of training data. Sign Language Recognition (SLR) is an important step in facilitating the communication among deaf people and the rest of society. Existing Persian sign language recognition systems are mainly restricted to static signs which are not very useful in everyday communications. In this study, a dynamic Persian sign language recognition system is presented. A collection of 1200 videos were captured from 12 individuals performing 20 dynamic signs with a simple white glove. The trajectory of the hands, along with hand shape information were extracted from each video using a simple region-growing technique. These time-varying trajectories were then modeled using Hidden Markov Model (HMM) with Gaussian probability density functions as observations. The performance of the system was evaluated in different experimental strategies. Signer-independent and signer-dependent experiments were performed on the proposed system and the average accuracy of 97.48% was obtained. The experimental results demonstrated that the performance of the system is independent of the subject and it can also perform excellently even with a limited number of training data.