Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

• Published in: IEEE transactions on intelligent transportation systems Vol. 24; no. 1; pp. 178 - 190
• Main Authors: Zhang, Weibin, Yan, Chen, Li, Xiaofeng, Fang, Liangliang, Wu, Yao-Jan, Li, Jun
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive control; Algorithms; Arterial traffic signal control; Control methods; Convergence; Coordination; Corridors; Deep learning; deep reinforcement learning; Delays; Green products; Heuristic algorithms; Machine learning; Methods; multi-agent reinforcement learning; Multiagent systems; Optimization; proximal policy optimization; reinforcement learning; Switches; Traffic congestion; Traffic control; Traffic delay; Traffic flow; Traffic intersections; Traffic signals; Training; Vehicle dynamics
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2022.3216203

Traffic congestion at signalized intersections often leads to serious impacts on adjacent intersections on a corridor. To enhance intersections' throughput efficiency, traffic signals are commonly coordinated across intersections. Traditional signal coordination methods control the adjacent intersections by setting a fixed phase offset. However, these traditional coordination methods may have poor adaptability to dynamic traffic conditions, which can cause additional congestion. To reduce arterial traffic delays, this paper develops an adaptive coordination control method based on multi-agent reinforcement learning (MARL). Most existing MARL-based methods rely on impractical assumptions to improve their performance in complex and dynamic traffic scenarios. To overcome these assumptions, this paper proposes a fully scalable MARL algorithm for arterial traffic signal coordination based on the proximal policy optimization algorithm. We apply a parameter-sharing training protocol to mitigate the slow convergence due to nonstationarity and to reduce computational requirements. In addition, a new action setting is designed by using the lead-lag phase sequence to simultaneously improve the implementation and coordination flexibility of the method. Extensive simulation experiments and comparisons with existing methods demonstrate that the proposed method performed stably in both simulated and real-world arterial corridors. Hence, the proposed signal coordination method can alleviate traffic congestion more effectively than existing traditional and MARL-based methods.