Robust task-based control policies for physics-based characters

• Published in: ACM transactions on graphics Vol. 28; no. 5; pp. 1 - 9
• Main Authors: Coros, Stelian, Beaudoin, Philippe, van de Panne, Michiel
• Format: Journal Article
• Language: English
• Published: 01.12.2009
• Subjects: Computer simulation; Controllers; Mathematical models; Policies; Position (location); Reinforcement; Tasks; Walking
• ISSN: 0730-0301; 1557-7368
• DOI: 10.1145/1618452.1618516

We present a method for precomputing robust task-based control policies for physically simulated characters. This allows for characters that can demonstrate skill and purpose in completing a given task, such as walking to a target location, while physically interacting with the environment in significant ways. As input, the method assumes an abstract action vocabulary consisting of balance-aware, step-based controllers. A novel constrained state exploration phase is first used to define a character dynamics model as well as a finite volume of character states over which the control policy will be defined. An optimized control policy is then computed using reinforcement learning. The final policy spans the cross-product of the character state and task state, and is more robust than the conrollers it is constructed from. We demonstrate real-time results for six locomotion-based tasks and on three highly-varied bipedal characters. We further provide a game-scenario demonstration.