Spelling it out: Real-time ASL fingerspelling recognition

This article presents an interactive hand shape recognition user interface for American Sign Language (ASL) finger-spelling. The system makes use of a Microsoft Kinect device to collect appearance and depth images, and of the OpenNI+NITE framework for hand detection and tracking. Hand-shapes corresp...

Full description

Saved in:
Bibliographic Details
Published in2011 IEEE International Conference on Computer Vision Workshops pp. 1114 - 1119
Main Authors Pugeault, N., Bowden, R.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.11.2011
Subjects
Feature extraction
Handicapped aids
Real time systems
Shape
User interfaces
Vectors
Vegetation
Online AccessGet full text
ISBN1467300624
9781467300629
DOI10.1109/ICCVW.2011.6130290

Cover

More Information
Summary:This article presents an interactive hand shape recognition user interface for American Sign Language (ASL) finger-spelling. The system makes use of a Microsoft Kinect device to collect appearance and depth images, and of the OpenNI+NITE framework for hand detection and tracking. Hand-shapes corresponding to letters of the alphabet are characterized using appearance and depth images and classified using random forests. We compare classification using appearance and depth images, and show a combination of both lead to best results, and validate on a dataset of four different users. This hand shape detection works in real-time and is integrated in an interactive user interface allowing the signer to select between ambiguous detections and integrated with an English dictionary for efficient writing.
ISBN:1467300624
9781467300629
DOI:10.1109/ICCVW.2011.6130290