Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides...

Full description

Saved in:
Bibliographic Details
Published inMachines (Basel) Vol. 10; no. 11; p. 1033
Main Authors Fan, Zihao, Xu, Yang, Kang, Yuhang, Luo, Delin
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.11.2022
Subjects
A3C
Air combat
Aircraft
Aircraft maneuvers
Algorithms
Analysis
asynchronous mechanism
Combat aircraft
Decision making
Deep learning
deep reinforcement learning
Flow velocity
Genetic algorithms
Kinematics
Machine learning
maneuver decision
Methods
Multiprocessing
Neural networks
Pitch (inclination)
Teaching methods
UCAV
Unmanned aerial vehicles
Unmanned aircraft
Online AccessGet full text

Cover

More Information
Summary:To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides are established. Then, considering the different state transition effects of various actions when the pitch angles of the UCAVs are different, the 10 state variables including the pitch angle, are taken as the state space. Combined with the air combat situation threat assessment index model, a two-layer reward mechanism combining internal reward and sparse reward is designed as the evaluation basis of reinforcement learning. Then, the neural network model of the full connection layer is built according to an Asynchronous Advantage Actor–Critic (A3C) algorithm. In the way of multi-threading, our UCAV keeps interactively learning with the environment to train the model and gradually learns the optimal air combat maneuver countermeasure strategy, and guides our UCAV to conduct action selection. The algorithm reduces the correlation between samples through multi-threading asynchronous learning. Finally, the effectiveness and feasibility of the method are verified in three different air combat scenarios.
ISSN:2075-1702
2075-1702
DOI:10.3390/machines10111033