Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

• Published in: Structural control and health monitoring Vol. 29; no. 10
• Main Authors: Li, Ning, Tang, Jichuan, Li, Zhong‐Xian, Gao, Xiuyu
• Format: Journal Article
• Language: English
• Published: Pavia Wiley Subscription Services, Inc 01.10.2022
• Subjects: Accuracy; Actuation; Adaptive algorithms; Algorithms; Benchmarks; Broadband; Compensation; Compensators; Complex systems; Control methods; Controllers; deep deterministic policy gradient algorithm; Dynamic tests; hybrid control; Machine learning; Performance assessment; Perturbation; real‐time hybrid simulation; Reinforcement; reinforcement learning; Robustness; Shake table tests; Simulation; Tracking; Uncertainty; Underwater; underwater shaking table; Water depth
• ISSN: 1545-2255; 1545-2263
• DOI: 10.1002/stc.3035

Summary The tracking performance of an actuation transfer system in a real‐time hybrid simulation (RTHS) frequently faces accuracy and robustness challenges under constraints and complicated environments with uncertainties. This study proposes a novel control approach based on the deep deterministic policy gradient algorithm in reinforcement learning (RL) combined with feedforward (FF) compensation, which emphasizes the implementation of shaking table control and substructure RTHS. The proposed method first describes the control plant within the RL environment. Then, the agent is trained offline to develop optimized control policies for interaction with the environment. A series of validation tests were conducted to assess the performance of the proposed method, starting with the dynamic testing of underwater shaking table control and then a virtual RTHS benchmark problem. For complex systems, such as controlling the underwater shaking table, the proposed algorithm, FF, and adaptive time series (ATS) compensation methods are compared under various water depths and motions. The results show better performance and wider broadband frequency applicability under different shaking table dynamic‐coupling effects. Next, a controller based on the proposed method was designed by extending the virtual RTHS via the configuration of the control plant and substructure division, as provided in the RTHS benchmark problem. The proposed RL controller also improved the tracking accuracy and robustness of conventional FF compensators against unmodeled dynamics and perturbation uncertainties. This controller can be extended to further advanced control strategies as a component of model‐based control methods.