Deceptive Path Planning via Reinforcement Learning with Graph Neural Networks
Deceptive path planning (DPP) is the problem of designing a path that hides its true goal from an outside observer. Existing methods for DPP rely on unrealistic assumptions, such as global state observability and perfect model knowledge, and are typically problem-specific, meaning that even minor ch...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
09.02.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Deceptive path planning (DPP) is the problem of designing a path that hides its true goal from an outside observer. Existing methods for DPP rely on unrealistic assumptions, such as global state observability and perfect model knowledge, and are typically problem-specific, meaning that even minor changes to a previously solved problem can force expensive computation of an entirely new solution. Given these drawbacks, such methods do not generalize to unseen problem instances, lack scalability to realistic problem sizes, and preclude both on-the-fly tunability of deception levels and real-time adaptivity to changing environments. In this paper, we propose a reinforcement learning (RL)-based scheme for training policies to perform DPP over arbitrary weighted graphs that overcomes these issues. The core of our approach is the introduction of a local perception model for the agent, a new state space representation distilling the key components of the DPP problem, the use of graph neural network-based policies to facilitate generalization and scaling, and the introduction of new deception bonuses that translate the deception objectives of classical methods to the RL setting. Through extensive experimentation we show that, without additional fine-tuning, at test time the resulting policies successfully generalize, scale, enjoy tunable levels of deception, and adapt in real-time to changes in the environment. |
---|---|
AbstractList | Deceptive path planning (DPP) is the problem of designing a path that hides its true goal from an outside observer. Existing methods for DPP rely on unrealistic assumptions, such as global state observability and perfect model knowledge, and are typically problem-specific, meaning that even minor changes to a previously solved problem can force expensive computation of an entirely new solution. Given these drawbacks, such methods do not generalize to unseen problem instances, lack scalability to realistic problem sizes, and preclude both on-the-fly tunability of deception levels and real-time adaptivity to changing environments. In this paper, we propose a reinforcement learning (RL)-based scheme for training policies to perform DPP over arbitrary weighted graphs that overcomes these issues. The core of our approach is the introduction of a local perception model for the agent, a new state space representation distilling the key components of the DPP problem, the use of graph neural network-based policies to facilitate generalization and scaling, and the introduction of new deception bonuses that translate the deception objectives of classical methods to the RL setting. Through extensive experimentation we show that, without additional fine-tuning, at test time the resulting policies successfully generalize, scale, enjoy tunable levels of deception, and adapt in real-time to changes in the environment. |
Author | Sadler, Brian M Fatemi, Michael Y Suttle, Wesley A |
Author_xml | – sequence: 1 givenname: Michael surname: Fatemi middlename: Y fullname: Fatemi, Michael Y – sequence: 2 givenname: Wesley surname: Suttle middlename: A fullname: Suttle, Wesley A – sequence: 3 givenname: Brian surname: Sadler middlename: M fullname: Sadler, Brian M |
BookMark | eNqNys0KgkAUQOEhCrLyHQZaC3bV0nW_iwqJ9jLINcfsjs2M-vpJ9ACtzuI7MzYmRThiDgTByotDgClzjal834f1BqIocNhlhzk2VnbIU2FLntaCSNKDd1LwG0oqlM7xhWT5GYX-Ui-H8ahFU_IrtlrUQ2yv9NMs2KQQtUH31zlbHvb37clrtHq3aGxWqVbTQBkkEEESxxAG_10f7NE_pg |
ContentType | Paper |
Copyright | 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni Edition) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_29252988243 |
IEDL.DBID | 8FG |
IngestDate | Tue Sep 24 22:57:42 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_29252988243 |
OpenAccessLink | https://www.proquest.com/docview/2925298824/abstract/?pq-origsite=%requestingapplication% |
PQID | 2925298824 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2925298824 |
PublicationCentury | 2000 |
PublicationDate | 20240209 |
PublicationDateYYYYMMDD | 2024-02-09 |
PublicationDate_xml | – month: 02 year: 2024 text: 20240209 day: 09 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2024 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.5110745 |
SecondaryResourceType | preprint |
Snippet | Deceptive path planning (DPP) is the problem of designing a path that hides its true goal from an outside observer. Existing methods for DPP rely on... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Changing environments Deception Distillation Graph neural networks Neural networks Observability (systems) Path planning Policies Real time State space models Testing time |
Title | Deceptive Path Planning via Reinforcement Learning with Graph Neural Networks |
URI | https://www.proquest.com/docview/2925298824/abstract/ |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1dS8MwFL3MFcG3-YUfcwT0NZSm3UqeBGe7InSUobC30aQ3IojOtdujv93c0OqDsKcSCiUJ5d6bk3PPAbgzNjJWqGKulC55pKuKy1gKrozSRqsgQEnNyfl8kr1ET8vxsgdZ1wtDtMouJrpAXX1qwsh9IcVYSFsPRn6pCAXQjX-__uLkH0X3rK2ZxgF4AWniUc94OvtFW8QktrVz-C_guiySDsAryjVujqGHHydw6MiXuj6F_BEduWSHrLAFGeuMhNjurWQLdNqm2sF4rJVDfWWEn7IZqU0z0tco3-3DEbrrM7hNk-dpxrs5rNr_pV79rS48h749-OMFsEiTGUhgDJKZEAmV2eKeOlFFrCVG4SUM933pav_razgSNkE7BrIcQr_ZbPHGJthGjdzejcB7SObFwo7y7-QHEbyHUA |
link.rule.ids | 786,790,12792,21416,33408,33779,43635,43840 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1NSwMxEB20RezNT7RWDeg1yGbTLjl5ULerdkuRCr0tm-xEBKm1W_v7zYSsHoSecgiEZFlmJi9v3gO4ti4yVqgTrrUpuTRVxVWiBNdWG2t0FKGi5uR8PMhe5dOsPwuAWx1olU1M9IG6-jSEkd8IJfpCuXpQ3i6-OLlG0etqsNDYhraMXeqkTvF0-IuxiEHiKub4X5j1uSPdg_akXOByH7ZwfgA7nnJp6kPI79FTStbIJq4MY419EFu_l-wFvaKp8eAdCyKob4xQUzYkjWlGqhrlhxs8jbs-gqv0YXqX8WYPRfhL6uLvTPExtNx1H0-ASUMWIJG1SBZCJE_mSnrqPxWJUSjjU-htWqm7efoSdrNpPipGj-PnM-gIl6I9B1n1oLVafuO5S7ErfeG_4w-dAYOz |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deceptive+Path+Planning+via+Reinforcement+Learning+with+Graph+Neural+Networks&rft.jtitle=arXiv.org&rft.au=Fatemi%2C+Michael+Y&rft.au=Suttle%2C+Wesley+A&rft.au=Sadler%2C+Brian+M&rft.date=2024-02-09&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |