Real Time Video To Text Conversion of Sign Language

• Published in: 2022 1st International Conference on Computational Science and Technology (ICCST) pp. 1 - 3
• Main Authors: Sharanyaa, S., Devi, S.V. Abinaya, Abinaya, S., Sakthi, J. Kiruthika, Niranjani, S.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 09.11.2022
• Subjects: Deep learning; Face recognition; Gesture recognition; LSTM; Mediapipe; Neural networks; Open CV; Reinforcement learning; Scientific computing; Sign Language; Tracking
• DOI: 10.1109/ICCST55948.2022.10040293

Sign Language is an non-verbal communication in which visible bodily actions are used to convey vital messages, either instead of or in conjunction with spoken words. Hands, face, and other parts of the body are used in sign language. Nonverbal body language, like merely expressive motions, proxemics, or joint displays of attention are distinct from gestures that convey specific messages. Gestures are culturally different and can communicate a wide range of meanings in different social and cultural contexts. In this work, the model will track the evolution of human behavior and recognize language patterns. Each movement of the joint, elbows and face must be identified. The goal of this research is to develop a deep learning model that can classify live video actions of human sign languages such as words and sentences. The classification accuracy of live action gesture types using LSTM is found to be higher than other state of art algorithms. It is expected that, if the LSTM approach is supplemented by the addition of continuous image sequential input, feature extraction methods and correct identification of video types, the success rate of the results produced will increase.