Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation

• Published in: Water (Basel) Vol. 12; no. 11; p. 3222
• Main Authors: Saliba, Sami M., Bowes, Benjamin D., Adams, Stephen, Beling, Peter A., Goodall, Jonathan L.
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 17.11.2020
• Subjects: algorithms; climate change; Design; flood control; Floods; infrastructure; issues and policy; modernization; rain; Sensors; stormwater; Stormwater management; uncertainty; urbanization; Virginia; Virginia
• ISSN: 2073-4441; 2073-4441
• DOI: 10.3390/w12113222

Flooding in many areas is becoming more prevalent due to factors such as urbanization and climate change, requiring modernization of stormwater infrastructure. Retrofitting standard passive systems with controllable valves/pumps is promising, but requires real-time control (RTC). One method of automating RTC is reinforcement learning (RL), a general technique for sequential optimization and control in uncertain environments. The notion is that an RL algorithm can use inputs of real-time flood data and rainfall forecasts to learn a policy for controlling the stormwater infrastructure to minimize measures of flooding. In real-world conditions, rainfall forecasts and other state information are subject to noise and uncertainty. To account for these characteristics of the problem data, we implemented Deep Deterministic Policy Gradient (DDPG), an RL algorithm that is distinguished by its capability to handle noise in the input data. DDPG implementations were trained and tested against a passive flood control policy. Three primary cases were studied: (i) perfect data, (ii) imperfect rainfall forecasts, and (iii) imperfect water level and forecast data. Rainfall episodes (100) that caused flooding in the passive system were selected from 10 years of observations in Norfolk, Virginia, USA; 85 randomly selected episodes were used for training and the remaining 15 unseen episodes served as test cases. Compared to the passive system, all RL implementations reduced flooding volume by 70.5% on average, and performed within a range of 5%. This suggests that DDPG is robust to noisy input data, which is essential knowledge to advance the real-world applicability of RL for stormwater RTC.