Using Reinforcement Learning to Minimize the Probability of Delay Occurrence in Transportation

• Published in: IEEE transactions on vehicular technology Vol. 69; no. 3; pp. 2424 - 2436
• Main Authors: Cao, Zhiguang, Guo, Hongliang, Song, Wen, Gao, Kaizhou, Chen, Zhenghua, Zhang, Le, Zhang, Xuexi
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Adaptation models; arriving on time; Computational modeling; Delays; Machine learning; Neural networks; Optimization; Performance enhancement; Probabilistic methods; Probability; Q-learning; Reinforcement learning; Roads; Shortest path problem; Shortest-path problems; Stochastic processes; Sustainable development; Time dependence; Traffic delay; transportation; Transportation networks; Transportation systems; vehicle routing
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2020.2964784

Reducing traffic delay is of crucial importance for the development of sustainable transportation systems, which is a challenging task in the studies of stochastic shortest path (SSP) problem. Existing methods based on the probability tail model to solve the SSP problem, seek for the path that minimizes the probability of delay occurrence, which is equal to maximizing the probability of reaching the destination before a deadline (i.e., arriving on time). However, they suffer from low accuracy or high computational cost. Therefore, we design a novel and practical Q-learning approach where the converged Q-values have the practical meaning as the actual probabilities of arriving on time so as to improve the accuracy of finding the real optimal path. By further adopting dynamic neural networks to learn the value function, our approach can scale well to large road networks with arbitrary deadlines. Moreover, our approach is flexible to implement in a time dependent manner, which further improves the performance of returned path. Experimental results on some road networks with real mobility data, such as Beijing, Munich and Singapore, demonstrate the significant advantages of the proposed approach over other methods.