Gesture modeling and recognition using finite state machines

• Published in: 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000) pp. 410 - 415
• Main Authors: Pengyu Hong, Turk, M., Huang, T.S.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2000
• Subjects: Applied sciences; Artificial intelligence; Automata; Computer science; control theory; systems; Decision support systems; Exact sciences and technology; Fiber reinforced plastics; Learning and adaptive systems; Pattern recognition. Digital image processing. Computational geometry; Performance evaluation; Computer vision; Alignment; Abstract machine; Tracking; Finite state machine; Learning (artificial intelligence); Classification; Gesture; Tracking(movable target); Automatic recognition; Real time
• ISBN: 0769505805; 9780769505800
• DOI: 10.1109/AFGR.2000.840667

We propose a state-based approach to gesture learning and recognition. Using spatial clustering and temporal alignment, each gesture is defined to be an ordered sequence of states in spatial-temporal space. The 2D image positions of the centers of the head and both hands of the user are used as features; these are located by a color-based tracking method. From training data of a given gesture, we first learn the spatial information and then group the data into segments that are automatically aligned temporally. The temporal information is further integrated to build a finite state machine (FSM) recognizer. Each gesture has a FSM corresponding to it. The computational efficiency of the FSM recognizers allows us to achieve real-time on-line performance. We apply this technique to build an experimental system that plays a game of "Simon Says" with the user.