Learning Actions from Observations

• Published in: IEEE robotics & automation magazine Vol. 17; no. 2; pp. 30 - 43
• Main Authors: Kruger, V., Herzog, D.L., Baby, S., Ude, A., Kragic, D.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2010; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Action grammars; Action primitives; Automation; Data mining; Data structures; Electrical Engineering, Electronic Engineering, Information Engineering; Elektroteknik och elektronik; Engineering and Technology; Hidden Markov models; Human; Human body tracking; Humans; Learning; Mathematical models; Motor primitives; Movements; Object detection; Parametric hidden Markov models; Representations; Robotics; Robots; Robotteknik och automation; Speech recognition; Speech synthesis; Supervised learning; Teknik; Trajectories; Unsupervised learning
• ISSN: 1070-9932; 1558-223X
• DOI: 10.1109/MRA.2010.936961

In the area of imitation learning, one of the important research problems is action representation. There has been a growing interest in expressing actions as a combination of meaningful subparts called action primitives. Action primitives could be thought of as elementary building blocks for action representation. In this article, we present a complete concept of learning action primitives to recognize and synthesize actions. One of the main novelties in this work is the detection of primitives in a unified framework, which takes into account objects and actions being applied to them. As the first major contribution, we propose an unsupervised learning approach for action primitives that make use of the human movements as well as object state changes. As the second major contribution, we propose using parametric hidden Markov models (PHMMs) for representing the discovered action primitives. PHMMs represent movement trajectories as a function of their desired effect on the object, and we will discuss 1) how these PHMMs can be trained in an unsupervised manner, 2) how they can be used for synthesizing movements to achieve a desired effect, and 3) how they can be used to recognize an action primitive and the effect from an observed acting human.