CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

• Published in: IEEE robotics and automation letters Vol. 7; no. 4; pp. 1 - 8
• Main Authors: Bellegarda, Guillaume, Ijspeert, Auke
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bioinspired Robot Learning; Deep learning; Foot; Generators; Legged locomotion; Legged Robots; Locomotion; Machine Learning for Robot Control; Oscillators; Quadrupedal robots; Railroad car couplings; Reinforcement learning; Robot sensing systems; Robustness; Sensory feedback
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2022.3218167

In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training, most notably to a dynamically added 13.75 kg load representing 115% of the nominal quadruped mass. We test several different observation spaces based on proprioceptive sensing and show that our framework is deployable with no domain randomization and very little feedback, where along with the oscillator states, it is possible to provide only contact booleans in the observation space.