Ensuring Threshold AoI for UAV-Assisted Mobile Crowdsensing by Multi-Agent Deep Reinforcement Learning With Transformer

• Published in: IEEE/ACM transactions on networking Vol. 32; no. 1; pp. 1 - 16
• Main Authors: Wang, Hao, Liu, Chi Harold, Yang, Haoming, Wang, Guoren, Leung, Kin K.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: AoI; Autonomous aerial vehicles; Crowdsensing; Data collection; Data integrity; Deep learning; multi-agent deep reinforcement learning; Multiagent systems; Optimization; Sensors; Trajectory; Trajectory planning; transformer; Transformers; UAV crowdsensing; Unmanned aerial vehicles
• ISSN: 1063-6692; 1558-2566
• DOI: 10.1109/TNET.2023.3289172

Unmanned aerial vehicle (UAV) crowdsensing (UCS) is an emerging data collection paradigm to provide reliable and high quality urban sensing services, with age-of-information (AoI) requirement to measure data freshness in real-time applications. In this paper, we explicitly consider the case to ensure that the attained AoI always stay within a specific threshold. The goal is to maximize the total amount of collected data from diverse Point-of-Interests (PoIs) while minimizing AoI and AoI threshold violation ratio under limited energy supplement. To this end, we propose a decentralized multi-agent deep reinforcement learning framework called "DRL-UCS(<inline-formula> <tex-math notation="LaTeX">\text{AoI}_{th}</tex-math> </inline-formula>)" for multi-UAV trajectory planning, which consists of a novel transformer-enhanced distributed architecture and an adaptive intrinsic reward mechanism for spatial cooperation and exploration. Extensive results and trajectory visualization on two real-world datasets in Beijing and San Francisco show that, DRL-UCS(<inline-formula> <tex-math notation="LaTeX">\text{AoI}_{th}</tex-math> </inline-formula>) consistently outperforms all nine baselines when varying the number of UAVs, AoI threshold and generated data amount in a timeslot.