Representation Reinforcement Learning-Based Dense Control for Point Following With State Sparse Sensing of 3-D Snake Robots

• Published in: IEEE/ASME transactions on mechatronics Vol. 30; no. 2; pp. 851 - 861
• Main Authors: Liu, Lixing, Liu, Jiashun, Guo, Xian, Huang, Wei, Fang, Yongchun, Hao, Jianye
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3-D snake robots; Algorithms; biomimetic robots; Crawlers; dense motion control; Motion control; Obstructions; Optimization; Process control; Real time; representation reinforcement learning (RRL); Representations; Robot control; Robot dynamics; Robot sensing systems; Robots; Sensors; Simulation; Smoothness; Snake robots; sparse state sensing; Training
• ISSN: 1083-4435; 1941-014X
• DOI: 10.1109/TMECH.2024.3465018

During robot movements, the environmental states often fail to update in real-time due to interference from various factors, such as obstacle obstructions, communication disruptions, etc., which commonly results in interruptions or even failures in motion control. To achieve dense motion control under sparse state sensing, an important challenge is to predict future multiple actions based on sparse states, which is hindered by the large and complex action search space. Unfortunately, limited research has been dedicated to addressing this challenge. Therefore, this article proposes a representation reinforcement learning (RRL) based solution, called Sparse-State to Dense-Actions Latent Control , designed to realize dense motion control of 3-D snake robots subject to sparse environmental state sensing, which guarantees satisfactory point following performance. In particular, by introducing a latent representation of multiple actions, the control policy optimizes latent actions to predict dense motion gaits, which significantly enhances training efficiency and motion performance. Meanwhile, to learn a compact latent variable model, three mechanisms are proposed to ensure its efficient training, semantic smoothness, and energy efficiency, facilitating exploration of the RL algorithm. To the best of our knowledge, this article provides the first solution that enables a 3-D snake robot to successfully accomplish point following tasks under sparse state sensing. Simulation and practical experiments confirm the effectiveness, robustness, and generalizability of the proposed algorithm, with all following errors less than 0.02 m.