A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models

• Published in: IEEE transactions on robotics Vol. 21; no. 1; pp. 47 - 57
• Main Authors: Bernardin, K., Ogawara, K., Ikeuchi, K., Dillmann, R.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.02.2005; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Computer science; control theory; systems; Control theory. Systems; Data gloves; Education; Educational robots; Exact sciences and technology; Grasping; Hidden Markov models; Hidden Markov models (HMMs); Humans; Manipulation; Programming by Demonstration (PbD); Robot programming; Robot sensing systems; Robotics; Robots; Sensor fusion; Sensors; Shape; user interfaces; Human; User interface; Sensor fusion; Hidden Markov model; Programming; Grasp reflex; Robot
• ISSN: 1552-3098; 1941-0468
• DOI: 10.1109/TRO.2004.833816

The Programming by Demonstration (PbD) technique aims at teaching a robot to accomplish a task by learning from a human demonstration. In a manipulation context, recognizing the demonstrator's hand gestures, specifically when and how objects are grasped, plays a significant role. Here, a system is presented that uses both hand shape and contact-point information obtained from a data glove and tactile sensors to recognize continuous human-grasp sequences. The sensor fusion, grasp classification, and task segmentation are made by a hidden Markov model recognizer. Twelve different grasp types from a general, task-independent taxonomy are recognized. An accuracy of up to 95% could be achieved for a multiple-user system.