Stand-alone air combat decision-making method based on curriculum type reinforcement learning

• Main Authors: HE YUHANG, LI YAN, WANG ZHONG
• Format: Patent
• Language: Chinese; English
• Published: 11.07.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FORADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORYOR FORECASTING PURPOSES; PHYSICS; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE,COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTINGPURPOSES, NOT OTHERWISE PROVIDED FOR

The invention relates to a stand-alone air combat decision-making method based on curriculum-type reinforcement learning, and the method comprises the steps: building a dynamic model of a combat aircraft in a three-dimensional space, and selecting a state space vector of the combat aircraft; for a full-dimensional observation environment, an action network and an evaluation network are designed based on a full-connection neural network, and a gradient back propagation algorithm of the neural network is designed by using a PPO reinforcement learning algorithm. In order to solve the problem that a traditional reward function cannot break through human experience, a course type reward function module is established, an air combat strategy is gradually learned from easy to difficult, and the problem that learning is difficult under sparse reward information is solved. An intelligent agent experience playback mechanism is introduced in the air combat process, so that an intelligent agent is always subjected to con