Unlocking Communication: YOLO Sign Language Detection System

• Published in: International Conference on Computing Communication Control and Automation (Online) pp. 1 - 7
• Main Authors: Verma, Shivam, Chandok, Vishesh, Gupte, Ayush, Badhiye, Sagarkumar S., Borkar, Pradnya, Agrawal, Pratik K.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 23.08.2024
• Subjects: Accuracy; Classification algorithms; Hand Gestures; Real-time systems; Robots; Sign language; Sign Language Recognition; Testing; Training; Webcams; YOLO
• ISSN: 2771-1358
• DOI: 10.1109/ICCUBEA61740.2024.10775264

One of the most important methods for helping deaf-mute persons communicate with is the detecting the hand gesture of deaf and mute community people. A sign language detection system tracks and recognizes the meaningful expressions of humans-made with their heads, arms, hands, fingers, etc. In this paper, we propose YOLO (You Only Look Once) based sign recognition method, focus on improving the traditional sign language recognition accuracy. The proposed system uses a camera to capture images of person hand gesture, then annotate the collected image with bounding boxes around the hand gesture and then processing of image frame is done by Roboflow [1] to detect and recognize the hand gestures. Hand gestures are then further classified and mapped to corresponding words and phrases in sign language using algorithms. Since there have been many existing sign language recognition systems which are designed using convolutional neural network, algorithms but they are not real-time detector. In this paper, we focused to create a real-time detection of hand gestures using a webcam. Even with a small dataset size, the system achieves a good degree of accuracy.