Versatile Control of Fluid-directed Solid Objects Using Multi-task Reinforcement Learning

• Published in: ACM transactions on graphics Vol. 42; no. 2; pp. 1 - 14
• Main Authors: Ren, Bo, Ye, Xiaohan, Pan, Zherong, Zhang, Taiyuan
• Format: Journal Article
• Language: English
• Published: New York, NY ACM 18.10.2022
• Subjects: Computing methodologies; Physical simulation; Fluid/solid coupling; optimal control; reinforcement learning
• ISSN: 0730-0301; 1557-7368
• DOI: 10.1145/3554731

We propose a learning-based controller for high-dimensional dynamic systems with coupled fluid and solid objects. The dynamic behaviors of such systems can vary across different simulators and the control tasks subject to changing requirements from users. Our controller features high versatility and can adapt to changing dynamic behaviors and multiple tasks without re-training, which is achieved by combining two training strategies. We use meta-reinforcement learning to inform the controller of changing simulation parameters. We further design a novel task representation, which allows the controller to adapt to continually changing tasks via hindsight experience replay. We highlight the robustness and generality of our controller on a row of dynamic-rich tasks, including scooping up solid balls from a water pool, in-air ball acrobatics using fluid spouts, and zero-shot transferring to unseen simulators and constitutive models. In all the scenarios, our controller consistently outperforms the plain multi-task reinforcement-learning baseline.