Dynamic Tool for American Sign Language Finger Spelling Interpreter

• Published in: 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN) pp. 596 - 600
• Main Authors: S.G., Prateek, J., Jagadeesh, R., Siddarth, Y., Smitha, Hiremath, P. G. Sunitha, Pendari, Neha Tarannum
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2018
• Subjects: American Sign Language (ASL); Assistive technology; CNN classifier; Computational modeling; Convolutional neural networks; Data models; Finger spelling interpreter; Gesture recognition; Image edge detection; Sign classification; Videos
• DOI: 10.1109/ICACCCN.2018.8748859

Sign language is a type of language that uses manual communication to convey meaningful messages to other people. This includes simultaneous employing of hand gestures, movement, orientation of the fingers, arms or body, and facial expressions to convey a speaker's ideas. American Sign Language is one of the popular sign language used by most of deaf and dumb people to communicate with each other. American Sign Language is also referred to as ASL. A real-time sign language translator is required for facilitating communication between the deaf community and the general public. We propose a system called Dynamic tool for American Sign Language (ASL) finger spelling interpreter which can consistently classify the letters a-z. The dataset consists of a set of American Sign Language videos. Our approach first converts the videos into frames and then pre-processes the frames to convert them into greyscale images. Then the Convolutional Neural Network (CNN) classifier is used for building the classification model which classifies the frames into 26 different classes representing 26 English alphabets. Finally, the evaluation of the classification model is carried out with test data providing the output in the form of text or voice. The cross-validation accuracy results of 98.66% is achieved from our approach.