Reinforcement Learning for Quantization of Boundary Control Inputs: A Comparison of PPO-based Strategies

• Published in: Chinese Control Conference pp. 1093 - 1098
• Main Authors: Wang, Yibo, Kang, Wen
• Format: Conference Proceeding
• Language: English
• Published: Technical Committee on Control Theory, Chinese Association of Automation 28.07.2024
• Subjects: Boundary stabilization; Deep reinforcement learning; Input quantization; Optimization; PD control; Quantization (signal); Simulation; The nonlinear Korteweg-de Vries equation
• ISSN: 1934-1768
• DOI: 10.23919/CCC63176.2024.10661946

This paper investigates the boundary stabilization problem for the Korteweg-de Vries (KdV) system with quantized control inputs via the deep reinforcement learning (DRL) approach. To examine the impact of different placements of the quantizer on stabilization performance, we discuss two scenarios: the quantizer placed in the environment and in the agent. In the case of 'introducing the quantizer into the agent', we further explore two variations: optimizing the parameters of the discretized continuous distribution and directly optimizing the parameters of the discrete distribution. Finally, simulation results demonstrate that the proposed proximal policy optimization (PPO)-based strategies can train DRL controllers that effectively stabilize the target system, with the approach directly learning the parameters of the discrete distribution achieving the highest stabilization efficiency among the quantization-based scenarios.