Sign Language Recognition Using Convolutional Neural Networks

• Published in: Computer Vision - ECCV 2014 Workshops pp. 572 - 578
• Main Authors: Pigou, Lionel, Dieleman, Sander, Kindermans, Pieter-Jan, Schrauwen, Benjamin
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing 2015
• Series: Lecture Notes in Computer Science
• Subjects: Convolutional neural network; Deep learning; Gesture recognition; Sign language recognition
• ISBN: 9783319161778; 3319161776
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-16178-5_40

There is an undeniable communication problem between the Deaf community and the hearing majority. Innovations in automatic sign language recognition try to tear down this communication barrier. Our contribution considers a recognition system using the Microsoft Kinect, convolutional neural networks (CNNs) and GPU acceleration. Instead of constructing complex handcrafted features, CNNs are able to automate the process of feature construction. We are able to recognize 20 Italian gestures with high accuracy. The predictive model is able to generalize on users and surroundings not occurring during training with a cross-validation accuracy of 91.7%. Our model achieves a mean Jaccard Index of 0.789 in the ChaLearn 2014 Looking at People gesture spotting competition.