XRL-FlexSBR: Multi-agent reinforcement learning-driven flexible SBR control with explainable performance guarantee under diverse influent conditions

• Published in: Journal of water process engineering Vol. 66; p. 105991
• Main Authors: Heo, SungKu, Nam, KiJeon, Kim, SangYoun, Yoo, ChangKyoo
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.09.2024
• Subjects: Explainable AI; Flexible systems; Multi-agent reinforcement learning; Multivariate control; Sequencing batch reactor (SBR); Multi-agent reinforcement learning; Sequencing batch reactor (SBR); Explainable AI; Flexible systems; Multivariate control
• ISSN: 2214-7144; 2214-7144
• DOI: 10.1016/j.jwpe.2024.105991

The sequencing batch reactor (SBR) process stands out for its small footprint and operational flexibility. However, the SBR process is highly nonlinear and subject to influent disturbances. In this study, we suggested an explainable multi-agent reinforcement learning (XRL) approach coupled with multi-agent reinforcement learning (MARL) and explainable AI (XAI); then, an XRL-driven flexible SBR control (XRL-FlexSBR) system was developed to conduct multivariate control the SBR process autonomously. Influent big datasets including biochemical oxygen demand (BOD) and total nitrogen (TN) were collected from the wastewater treatment plants (WWTPs) of South Korea. Then, the Gaussian mixture model was utilized to cluster the diverse influent conditions and the SBR mechanistic model was developed. A game abstraction method based on a two-stage attention network (G2ANET), one of MARL algorithms, was employed to manipulate dissolved oxygen (DO) and extra carbon injection (EC) controllers in the SBR process; furthermore, layer-wise relevance propagation (LRP) of explainable AI (XAI) technique was utilized to evaluate a control performance guarantee by G2ANET. The results verified that XRL-FlexSBR can control the DO and EC controllers in the SBR process while reducing the energy consumption by 4.93 % on average and maintaining effluent quality criteria across the diverse influent conditions. Furthermore, XAI explained that the improved control performance of XRL-FlexSBR agents is attributed to their understanding of the mechanism underlying SBR operations without human intervention. Hence the proposed XRL-FlexSBR can flexibly control the SBR to improve sustainability and profitability under varying influent conditions. [Display omitted] •Explainable MARL-driven flexible SBR control (XRL-FlexSBR) was proposed.•Bigdata of influent conditions was clustered as scenario by Gaussian mixture model.•Mechanistic model was developed for phase transition in SBR operation.•XRL-FlexSBR control SBR process flexibly while reducing aeration energy by 4.93 %.•Control performance of XRL-FlexSBR can be guaranteed by explainable AI (XAI).