Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm

• Published in: Drone systems and applications Vol. 11; pp. 1 - 17
• Main Authors: Millar, Richard C., Hashemi, Leila, Mahmoodi, Armin, Meyer, Robert Walter, Laliberte, Jeremy
• Format: Journal Article
• Language: English
• Published: NRC Research Press 01.01.2023; Canadian Science Publishing
• Subjects: Algorithms; Bayesian belief network; Control systems; Drone aircraft; LIDAR sensor; Methods; multi-objective reinforcement algorithm; Reinforcement learning (Machine learning); Remote sensing; trajectory optimization; unmanned aerial vehicle (UAV); United States
• ISSN: 2564-4939; 2564-4939
• DOI: 10.1139/dsa-2022-0043

This paper presents and assesses the feasibility and potential of a novel concept: the operation of multiple Unmanned Aerial Vehicles (UAVs) commanded and supported by a manned “Tender” air vehicle carrying a pilot and flight manager(s). The “Tender” is equipped to flexibly and economically monitor and manage multiple diverse UAVs over otherwise inaccessible terrain through wireless communication. The proposed architecture enables operations and analysis supported by the means to detect, assess, and accommodate change and hazards on the spot with effective human observation and coordination. Further, this paper seeks to find the optimal trajectories for UAVs to collect data from sensors in a predefined continuous space. We formulate the path-planning problem for a cooperative, and a diverse swarm of UAVs tasked with optimizing multiple objectives simultaneously with the goal of maximizing accumulated data within a given flight time within cloud data processing constraints as well as minimizing the probable imposed risk during UAVs mission. The risk assessment model determines risk indicators using an integrated Specific Operation Risk Assessment—Bayesian belief network approach, while its resultant analysis is weighted through the analytic hierarchy process ranking model. To this end, as the problem is formulated as a convex optimization model, and we propose a low complexity multi-objective reinforcement learning (MORL) algorithm with a provable performance guarantee to solve the problem efficiently. We show that the MORL architecture can be successfully trained and allows each UAV to map each observation of the network state to an action to make optimal movement decisions. This proposed network architecture enables the UAVs to balance multiple objectives. Estimated MSE measures show that the algorithm produced decreasing errors in the learning process with increasing epoch number.