Hierarchical Reinforcement Learning for Plantwide Control

• Published in: Computer Aided Chemical Engineering Vol. 53; pp. 1639 - 1644
• Main Authors: Bloor, Maximilian, Ahmed, Akhil, Kotecha, Niki, Tsay, Calvin, Mercangöz, Mehmet, del Rio-Chanona, Antonio
• Format: Book Chapter
• Language: English
• Published: 2024
• ISBN: 9780443288241; 0443288240
• ISSN: 1570-7946
• DOI: 10.1016/B978-0-443-28824-1.50274-X

In this paper, we introduce a novel hierarchical reinforcement learning algorithm for plant-wide control, combining a high-level artificial neural network with low-level PID controllers. We evaluate the algorithm's performance using a computational case study focused on setpoint tracking, noise control, and disturbance rejection. Comparative analysis with derivative-free optimization, multiloop relay tuning, and a nonlinear model predictive controller demonstrates that the hierarchical reinforcement learning algorithm consistently outperforms traditional PID tuning methods in terms of integral square error. However, the NMPC excels in scenarios where manipulating other system units enhances setpoint tracking beyond PID capabilities. We also assess the controllers' robustness through a parametric mismatch analysis, simulating reactor cooling jacket fouling and reactor catalyst degradation. This analysis highlights that the hierarchical reinforcement learning algorithm's lesser dependence from an accurate model gives it an advantage over NMPC when a plant-model mismatch exists.