A graph deep reinforcement learning traffic signal control for multiple intersections considering missing data

• Published in: IEEE transactions on vehicular technology pp. 1 - 13
• Main Authors: Xu, Dongwei, Yu, Zefeng, Liao, Xiangwang, Guo, Haifeng
• Format: Journal Article
• Language: English
• Published: IEEE 14.08.2024
• Subjects: Aerospace electronics; Deep reinforcement learning; Estimation; Feature extraction; Generative adversarial network; Graph deep reinforcement learning; Roads; Traffic signal control; Training; Two-stage attention network; Vehicle dynamics
• ISSN: 0018-9545
• DOI: 10.1109/TVT.2024.3444475

Efficient traffic signal control (TSC) for multiple intersections is an important way to solve traffic congestion. With the development of deep reinforcement learning (DRL), an increasing number of DRL methods are applied to TSC. However, the prevailing methods in DRL-based TSC does not adequately address the issue of missing data within the agent state space. Furthermore, these methods often insufficiently account for the intricate interactions and relationships among the agents involved. Therefore, a graph deep reinforcement learning traffic signal control for multiple intersections considering missing data is proposed in this paper. Firstly, we propose an agent state space estimation method based on wasserstein generative adversarial network (WGAN). This method is adept at addressing the issue of diverse types of missing data within the state space to ensure its integrity. Secondly, we propose a graph deep reinforcement learning based on two-stage attention network and GraphSage (TAGGRL) to improve TSC efficiency for multiple intersections. A dynamic interaction graph based on two-stage attention network is constructed to facilitate effective interactions among agents. GraphSAGE is constructed for aggregating multiagent state features. Then, the decision network outputs Q-values based on the extracted features, which guide agents in executing phase-specific actions to enhance the smoothness of multiple intersections. Finally, the experimental results confirm that the agent state space estimation based on WGAN successfully solves the problem of missing data within the state space, thereby enhancing the robustness of TSC for multiple intersections. Furthermore, the TAGGRL model surpasses the baseline model in terms of TSC efficiency for multiple intersections.