A Safety-Enhanced Reinforcement Learning-Based Decision-Making Method by the Dimensionality Reduction Monte Carlo Tree Search

Left-turning at unsignalized intersections poses significant challenges for automated vehicles. On this regard, Deep Reinforcement Learning (DRL) methods can achieve better traffic efficiency and success rate than rule-based methods, but they occasionally lead to collisions. This paper proposes a sa...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on vehicular technology pp. 1 - 14
Main Authors Zhang, Lei, Cheng, Shuhui, Wang, Zhenpo, Liu, Jizheng, Wang, Mingqiang
Format Journal Article
LanguageEnglish
Published IEEE 15.07.2024
Subjects
Online AccessGet full text

Cover

Loading…
Abstract Left-turning at unsignalized intersections poses significant challenges for automated vehicles. On this regard, Deep Reinforcement Learning (DRL) methods can achieve better traffic efficiency and success rate than rule-based methods, but they occasionally lead to collisions. This paper proposes a safety-enhanced method that integrates the DRL and the Dimensionality Reduction Monte Carlo Tree Search (DRMCTS) algorithm to achieve safety-enhanced trajectory planning at unsignalized intersections. First, DRMCTS is employed to address the partially observable Markov decision process problem. Through dimensionality reduction, it effectually enhances computational efficiency and problem-solving performance. Then a unified framework is introduced by simultaneously implementing DRL and the Gaussian Mixture Model Hidden Markov Model (GMM-HMM) in real-time. DRL determines actions in the current state while GMM-HMM identifies the turning intentions of surrounding vehicles (SVs). Under safe driving conditions, DRL makes decisions and outputs longitudinal acceleration with optimized ride comfort and traffic efficiency. When unsafe driving conditions are detected, DRMCTS would be activated to generate a collision-free trajectory to enhance the ego vehicle's driving safety. Through comprehensive simulations, the proposed scheme demonstrates superior traffic efficiency and reduced collision rates at unsignalized intersections with multiple SVs present.
AbstractList Left-turning at unsignalized intersections poses significant challenges for automated vehicles. On this regard, Deep Reinforcement Learning (DRL) methods can achieve better traffic efficiency and success rate than rule-based methods, but they occasionally lead to collisions. This paper proposes a safety-enhanced method that integrates the DRL and the Dimensionality Reduction Monte Carlo Tree Search (DRMCTS) algorithm to achieve safety-enhanced trajectory planning at unsignalized intersections. First, DRMCTS is employed to address the partially observable Markov decision process problem. Through dimensionality reduction, it effectually enhances computational efficiency and problem-solving performance. Then a unified framework is introduced by simultaneously implementing DRL and the Gaussian Mixture Model Hidden Markov Model (GMM-HMM) in real-time. DRL determines actions in the current state while GMM-HMM identifies the turning intentions of surrounding vehicles (SVs). Under safe driving conditions, DRL makes decisions and outputs longitudinal acceleration with optimized ride comfort and traffic efficiency. When unsafe driving conditions are detected, DRMCTS would be activated to generate a collision-free trajectory to enhance the ego vehicle's driving safety. Through comprehensive simulations, the proposed scheme demonstrates superior traffic efficiency and reduced collision rates at unsignalized intersections with multiple SVs present.
Author Liu, Jizheng
Wang, Zhenpo
Cheng, Shuhui
Zhang, Lei
Wang, Mingqiang
Author_xml – sequence: 1
  givenname: Lei
  orcidid: 0000-0002-1763-0397
  surname: Zhang
  fullname: Zhang, Lei
  organization: Advanced Technology Research Institute, Beijing Institute of Technology, Beijing, China
– sequence: 2
  givenname: Shuhui
  surname: Cheng
  fullname: Cheng, Shuhui
  organization: National Engineering Research Center for Electric Vehicles, Beijing Institute of Technology, Beijing, China
– sequence: 3
  givenname: Zhenpo
  orcidid: 0000-0002-1396-906X
  surname: Wang
  fullname: Wang, Zhenpo
  organization: National Engineering Research Center for Electric Vehicles, Beijing Institute of Technology, Beijing, China
– sequence: 4
  givenname: Jizheng
  orcidid: 0000-0002-4350-0879
  surname: Liu
  fullname: Liu, Jizheng
  organization: National Engineering Research Center for Electric Vehicles, Beijing Institute of Technology, Beijing, China
– sequence: 5
  givenname: Mingqiang
  surname: Wang
  fullname: Wang, Mingqiang
  organization: National Engineering Research Center for Electric Vehicles, Beijing Institute of Technology, Beijing, China
BookMark eNpNkMtOAjEUhhuDiYDuXbjoCxR7HegSAS8JxEQmbielc-pUoWM6dTEL390SWLg6-c9_WXwjNAhtAIRuGZ0wRvV9-V5OOOVyIiSXiosLNGRaaKKF0gM0pJTNiFZSXaFR131mKaVmQ_Q7x1vjIPVkFRoTLNT4DXxwbbRwgJDwGkwMPnyQB9NlcwnWd74NZGO-8hdvIDVtjXc9Tg3gpc-do232PvV5qf6xKUu8aUMCvDBx3-IyAuBtnrXNNbp0Zt_BzfmOUfm4KhfPZP369LKYr4ktuCY7R1ldSKqt4cCV2ylmKCsKqxm1lgk15U4qJ_RMUMaMlYWeSnBTcGCd4rUYI3qatbHtugiu-o7-YGJfMVod6VWZXnWkV53p5crdqeIB4F9caT3LXP8AKzluvg
CODEN ITVTAB
ContentType Journal Article
DBID 97E
RIA
RIE
AAYXX
CITATION
DOI 10.1109/TVT.2024.3424523
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005-present
IEEE All-Society Periodicals Package (ASPP) 1998-Present
IEEE Xplore
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1939-9359
EndPage 14
ExternalDocumentID 10_1109_TVT_2024_3424523
10599819
Genre orig-research
GroupedDBID -~X
.DC
0R~
29I
4.4
5GY
6IK
97E
AAIKC
AAJGR
AAMNW
AASAJ
ABQJQ
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AKJIK
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
HZ~
IFIPE
IPLJI
JAVBF
LAI
MS~
O9-
OCL
P2P
RIA
RIE
RIG
RNS
RXW
TAE
TN5
3EH
5VS
AAYOK
AAYXX
AETIX
AI.
AIBXA
ALLEH
CITATION
EJD
H~9
IAAWW
IBMZZ
ICLAB
IFJZH
M43
VH1
ID FETCH-LOGICAL-c629-bf01d6409ca2e25fb51a0166c910cc13572f45f3983011ac46974ef7efecf52d3
IEDL.DBID RIE
ISSN 0018-9545
IngestDate Wed Jul 24 12:28:29 EDT 2024
Wed Jul 24 06:47:56 EDT 2024
IsPeerReviewed true
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c629-bf01d6409ca2e25fb51a0166c910cc13572f45f3983011ac46974ef7efecf52d3
ORCID 0000-0002-1396-906X
0000-0002-4350-0879
0000-0002-1763-0397
PageCount 14
ParticipantIDs ieee_primary_10599819
crossref_primary_10_1109_TVT_2024_3424523
PublicationCentury 2000
PublicationDate 20240715
PublicationDateYYYYMMDD 2024-07-15
PublicationDate_xml – month: 7
  year: 2024
  text: 20240715
  day: 15
PublicationDecade 2020
PublicationTitle IEEE transactions on vehicular technology
PublicationTitleAbbrev TVT
PublicationYear 2024
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0014491
Score 2.486467
Snippet Left-turning at unsignalized intersections poses significant challenges for automated vehicles. On this regard, Deep Reinforcement Learning (DRL) methods can...
SourceID crossref
ieee
SourceType Aggregation Database
Publisher
StartPage 1
SubjectTerms Automated vehicles
Decision making
deep reinforcement learning
Hidden Markov models
partially observable Markov decision process
Planning
Safety
Switches
Trajectory
Turning
turning intention recognition
Title A Safety-Enhanced Reinforcement Learning-Based Decision-Making Method by the Dimensionality Reduction Monte Carlo Tree Search
URI https://ieeexplore.ieee.org/document/10599819
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3NS8MwFA-6kx78nDi_yMGLh3Rr2rTNce6DIXQHrbJbadMXBaWT0R0m-L_7knYyBcFbCUkJeWnze3nv_X6EXGsFCCKgYCKT6KBwrlgkRMRUz88CLwyVZ4vE4mkwefTvZmLWFKvbWhgAsMln4JhHG8sv5mpprsq6BgvIyJB8bodS1sVa3yED32_k8Vz8ghEXrGOSPdlNnhL0BLnveCbOx70fZ9CGqIo9U8b7ZLqeTZ1K8uosq9xRH7-IGv893QOy16BL2q-3wyHZgvKI7G5wDh6Tzz59yDRUKzYqX2z4n96DpU9V9qaQNoyrz-wWD7iCDhsRHhZb3SoaW8lpmq8oQkc6NOIANbEHwnl8U1Gz0dLYsF7RQbZ4m9NkAUDrvOY2ScajZDBhjQYDUwGXLNc9twjQB1QZBy50LtwMQWKgEGUo5Xoi5NoX2pOR-VFkCp3t0AcdggalBS-8E9Iq5yWcEootMgq1ttJI2pWyCAPsYSKdHATkHXKzNkr6XjNtpNZD6ckUDZgaA6aNATukbZZ7o1-90md_tJ-THTPc3Mm64oK0qsUSLhFMVPmV3URf4YzGuQ
link.rule.ids 315,783,787,799,27938,27939,55088
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV07T8MwELZQGYCBZxHl6YGFwW3jxEk8lj5UoOkAAXWLEucMEqhFVTsUif_O2UlRQUJiiyzLsnx2_J3v7vsIudQKEERAzkQq0UHhXLFQiJCpppf6bhAo1xaJRUO__-jdjsSoLFa3tTAAYJPPoG4-bSw_n6i5eSprGCwgQ0PyuS4MsCjKtb6DBp5XCuQ5eIYRGSyjkk3ZiJ9i9AW5V3dNpI-7P26hFVkVe6v0dshwOZ8imeS1Pp9ldfXxi6rx3xPeJdslvqStYkPskTUY75OtFdbBA_LZog-phtmCdccvNgGA3oMlUFX2rZCWnKvP7BqvuJx2ShkeFlnlKhpZ0WmaLSiCR9ox8gAFtQcCehwpL_hoaWR4r2g7nb5NaDwFoEVmc5XEvW7c7rNShYEpn0uW6aaT--gFqpQDFzoTToow0VeIM5RyXBFw7QntytD8KlKF7nbggQ5Ag9KC5-4hqYwnYzgiFFtkGGhtxZG0I2Ue-NjDxDo5CMhq5GpplOS94NpIrI_SlAkaMDEGTEoD1kjVLPdKv2Klj_9ovyAb_TgaJIOb4d0J2TRDmRdaR5ySymw6hzOEFrPs3G6oL3SXygY
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Safety-Enhanced+Reinforcement+Learning-Based+Decision-Making+Method+by+the+Dimensionality+Reduction+Monte+Carlo+Tree+Search&rft.jtitle=IEEE+transactions+on+vehicular+technology&rft.au=Zhang%2C+Lei&rft.au=Cheng%2C+Shuhui&rft.au=Wang%2C+Zhenpo&rft.au=Liu%2C+Jizheng&rft.date=2024-07-15&rft.issn=0018-9545&rft.eissn=1939-9359&rft.spage=1&rft.epage=14&rft_id=info:doi/10.1109%2FTVT.2024.3424523&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TVT_2024_3424523
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9545&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9545&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9545&client=summon