SEGAC: Sample Efficient Generalized Actor Critic for the Stochastic On-Time Arrival Problem

This paper studies the stochastic on-time arrival (SOTA) problem in transportation networks and introduces a novel reinforcement learning-based algorithm, namely sample efficient generalized actor critic (SEGAC). Different from almost all canonical SOTA solutions, which are usually computationally e...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on intelligent transportation systems Vol. 25; no. 8; pp. 10190 - 10205
Main Authors Guo, Hongliang, He, Zhi, Sheng, Wenda, Cao, Zhiguang, Zhou, Yingjie, Gao, Weinan
Format Journal Article
LanguageEnglish
Published IEEE 01.08.2024
Subjects
Gaussian distribution
Generalized actor critic
Navigation
Optimization
Real-time systems
Reliability
Routing
sample efficiency
stochastic on-time arrival (SOTA)
Transportation
variance reduction
Online AccessGet full text

Cover

More Information
Summary:This paper studies the stochastic on-time arrival (SOTA) problem in transportation networks and introduces a novel reinforcement learning-based algorithm, namely sample efficient generalized actor critic (SEGAC). Different from almost all canonical SOTA solutions, which are usually computationally expensive and lack generalizability to unforeseen destination nodes, SEGAC offers the following appealing characteristics. SEGAC updates the ego vehicle's navigation policy in a sample efficient manner, reduces the variance of both value network and policy network during training, and is automatically adaptive to new destinations. Furthermore, the pre-trained SEGAC policy network enables its real-time decision-making ability within seconds, outperforming state-of-the-art SOTA algorithms in simulations across various transportation networks. We also successfully deploy SEGAC to two real metropolitan transportation networks, namely Chengdu and Beijing, using real traffic data, with satisfying results.
ISSN:1524-9050
1558-0016
DOI:10.1109/TITS.2024.3361445