Real-Time Hand Sign Language Translation: Text and Speech Conversion

• Published in: 2024 7th International Conference on Circuit Power and Computing Technologies (ICCPCT) Vol. 1; pp. 488 - 493
• Main Authors: P, Jeevanandham, A, George Britt, A, Hariharan, G, Keerthana
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.08.2024
• Subjects: Computer vision; Convolutional Neural Network (CNN); Human computer interaction; ISL-Indian sign language; Machine learning algorithms; Medieapipe; random forest; Sign language; Sign Language Recognition (SLR); Streaming media; Webcams; Word level Sign Language Recognition (WSLR); YOLO; You Only Look Once (YOLO)
• DOI: 10.1109/ICCPCT61902.2024.10673038

A real-time system that can decipher sign language from a live webcam stream is presented by the Sign Language Conversion Project. By using the Media Pipe library's landmark identification capabilities, the project extracts crucial data from every frame, including hand landmarks. Following detection, the landmark coordinates are gathered and saved in a CSV file for later examination. This landmark data is used to train a Random Forest algorithm, which uses machine learning techniques to categorize various sign language patterns. The trained model predicts the sign language class and its probability in real-time during the processing of the webcam feed. Users get instant access to the subject's sign language cues by superimposing the results over the live video feed. This work demonstrates the integration of computer vision and machine learning techniques to assess and comprehend nonverbal communication, with possible implications in human-computer interaction.