Deep Learning Based Classification of Single-Hand South Indian Sign Language Gestures

• Published in: Journal of Electrical Systems Vol. 20; no. 1s; pp. 702 - 711
• Main Authors: Yakkundimath, Rajesh, Badiger, Ramesh, Malvade, Naveen
• Format: Journal Article
• Language: English
• Published: Paris Engineering and Scientific Research Groups 28.03.2024
• Subjects: Accuracy; Algorithms; Classification; Deep learning; Disabilities; Disorders; Gesture recognition; Machine learning; Speech; Speech disorders; Verbal communication
• ISSN: 1112-5209
• DOI: 10.52783/jes.813

Gesture recognition is a branch of computer science and language technology dedicated to utilizing mathematical algorithms for the analysis of human gestures. Within the realm of non-verbal communication, the pivotal role of human arm movements and gestures remains a focal point. This research introduces advanced multi-stream deep transfer learning models tailored for identifying signs from South Indian languages, specifically Kannada, Tamil, and Telugu. The primary aim is to offer support to individuals encountering speech disorders or disabilities. The key deep transfer learning models utilized include Inception-V3, VGG-16, and ResNet-50, which have been modified and improved to attain heightened classification efficacy. The dataset comprises 35,000 images capturing single-hand gestures. In the realm of models, Inception-V3 exhibits the utmost test accuracy, standing at 91.45%, alongside a validation accuracy of 93.45% when tasked with the classification of single-hand gesture images across thirty-five distinct categories. The importance of this study lies in its prospective utility for creating an automated system that can support and improve the functional capabilities of individuals facing speech disorders or disabilities.