An AI-Based Detection System for Mudrabharati: A Novel Unified Fingerspelling System for Indic Scripts

• Published in: Text, Speech, and Dialogue Vol. 12848; pp. 425 - 434
• Main Authors: Amal Jude Ashwin, F., Chakravarthy, V. Srinivasa, Kopparapu, Sunil Kumar
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Fingerspelling; Finite state machine; Hand pose estimator; Indic scripts; Multilayer perceptron; Sign Language; Text entry software
• ISBN: 303083526X; 9783030835262
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-83527-9_36

Sign Language (SL) is a potential tool for communication in the hearing and speech-impaired community. As individual words cannot be communicated accurately using the SL gestures, fingerspelling is adopted to spell out names of people and places. Due to rich vocabulary and diversity in Indic scripts, and the abugida nature of Indic scripts that distinguish them from a prominent world script like the Roman script, it is cumbersome to use American Sign Language (ASL) convention for fingerspelling in Indian languages. Moreover, due to the existence of 10 major scripts in India, it is a futile task to develop a separate fingerspelling convention for each individual Indic script based on the geometry of the characters. In this paper, we propose a novel and unified fingerspelling system known as Mudrabharati for Indic scripts. The gestures of Mudrabharati are constructed based on the phonetics of Indian scripts and not the geometry of the glyphs that compose the individual characters. Unlike ASL that utilizes just one hand, Mudrabharati uses both the hands - one for consonants and the other for vowels; swarayukta aksharas (Consonant-Vowel combinations) are gestured by using both the hands. An Artificial Intelligence (AI) based recognition system for Mudrabharati that returns the character in Devanagari and Tamil scripts is developed.