Design of Reinforcement Learning Guidance Law for Antitorpedo Torpedoes

• Published in: International Journal of Aerospace Engineering Vol. 2025; no. 1
• Main Authors: Wang, Zhong, Wen, Zhiwen, Cui, Weitong, Zhou, Daming, Wang, Pei
• Format: Journal Article
• Language: English
• Published: New York John Wiley & Sons, Inc 01.01.2025; Wiley
• Subjects: Accuracy; Acoustics; Algorithms; Defense; Design; Guidance (motion); Kinematics; Laws, regulations and rules; Machine learning; Methods; Missiles; R&D; Research & development; Velocity
• ISSN: 1687-5966; 1687-5974
• DOI: 10.1155/ijae/5146939

Guidance law design is a critical technology that directly influences the interception performance of antitorpedo torpedoes. In response to the performance degradation of classic proportional guidance laws and derivative guidance laws when intercepting high‐speed underwater targets and the significant impact of the proportional coefficient on interception miss distance, this paper proposes an intelligent guidance law. Based on the proportional guidance interception law, the law incorporates a variable proportional coefficient based on the deep Q‐network (DQN) algorithm from deep reinforcement learning. Integrating engineering design, the intelligent guidance law for antitorpedo torpedoes proposed in this article selects the rate of change in the line‐of‐sight angle as the state variable, designs a reward function based on interception results, and designs a discretized behavior space based on the commonly used proportional guidance coefficient selection range. The greedy algorithm and temporal difference learning are employed to train the DQN, and the optimal proportional guidance coefficient is selected from the DQN by the real‐time state of the torpedo. The feasibility of the proposed guidance law was verified through simulation experiments. The interception effects of the proposed intelligent guidance law and the fixed coefficient proportional guidance law were compared and analyzed in typical situations. The results demonstrated that the reinforcement learning guidance law was significantly superior to the traditional proportional guidance law in terms of miss distance, maneuvering ability consumption, and ballistic straightness and had stronger robustness. Furthermore, the intelligent guidance law for antitorpedo torpedoes proposed in this article enables antitorpedo torpedoes to make autonomous decisions based on the battlefield situation.