Adaptive network traffic control with approximate dynamic programming based on a non-homogeneous Poisson demand model

• Published in: Transportmetrica. (Abingdon, Oxfordshire, UK) Vol. 12; no. 1
• Main Authors: Chen, Siqi, Lü, Xing
• Format: Journal Article
• Language: English
• Published: Abingdon Taylor & Francis 31.12.2024; Taylor & Francis Ltd
• Subjects: Adaptive algorithms; Adaptive control; approximate dynamic programming; cell transmission method; Communications traffic; Demand; Dynamic programming; Markov processes; Non-homogeneous Poisson process; Optimization; Random numbers; Recursive functions; temporal difference learning; Traffic control; Traffic flow; Traffic models; traffic signal control
• ISSN: 2168-0566; 2168-0582
• DOI: 10.1080/21680566.2024.2336029

In this study, we develop a stochastic dynamic traffic-flow model subject to practical restrictions under the non-homogeneous Poisson vehicle arrival process. Using the cell transmission strategy, we establish traffic dynamics between two intersections. We also discuss simulating the random demand of source links from an estimated intensity function. Additionally, we propose an algorithm to optimize time interval division for aggregated data, aiming to enhance estimation performance. We explore applying our traffic flow model to the adaptive traffic network management problem, which is formulated as a Markov decision process. Leveraging approximate dynamic programming with recursive least squares-temporal difference learning, we achieve adaptive optimal policies. To validate our approach, we conduct a series of numerical experiments with random demands. The results of non-homogeneous Poisson demand conducted using random numbers and a real-word dataset indicate high efficiency with the piecewise constant, I-SMOOTH, and MNO-PQRS estimators. Compared to the Webster and Max-pressure control systems, our proposed approximate dynamic programming-based model exhibits superior stability and applicability.