Enhanced Deep Deterministic Policy Gradient Algorithm Using Grey Wolf Optimizer for Continuous Control Tasks

• Published in: IEEE access Vol. 11; pp. 139771 - 139784
• Main Authors: Sumiea, Ebrahim Hamid Hasan, Abdulkadir, Said Jadid, Ragab, Mohammed Gamal, Al-Selwi, Safwan Mahmood, Fati, Suliamn Mohamed, AlQushaibi, Alawi, Alhussian, Hitham
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aerospace electronics; Algorithms; Artificial neural networks; Automobiles; Control tasks; Deep deterministic policy gradient; Deep learning; deep reinforcement learning; Games; grey wolf optimization; Heuristic methods; hyperparameters optimization; Machine learning; Metaheuristics; Optimal control; Optimization; Task analysis; Training
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2023.3341507

Deep Reinforcement Learning (DRL) allows agents to make decisions in a specific environment based on a reward function, without prior knowledge. Adapting hyperparameters significantly impacts the learning process and time. Precise estimation of hyperparameters during DRL training poses a major challenge. To tackle this problem, this study utilizes Grey Wolf Optimization (GWO), a metaheuristic algorithm, to optimize the hyperparameters of the Deep Deterministic Policy Gradient (DDPG) algorithm for achieving optimal control strategy in two simulated Gymnasium environments provided by OpenAI. The ability to adapt hyperparameters accurately contributes to faster convergence and enhanced learning, ultimately leading to more efficient control strategies. The proposed DDPG-GWO algorithm is evaluated in the 2DRobot and MountainCarContinuous simulation environments, chosen for their ease of implementation. Our experimental results reveal that optimizing the hyperparameters of the DDPG using the GWO algorithm in the Gymnasium environments maximizes the total rewards during testing episodes while ensuring the stability of the learning policy. This is evident in comparing our proposed DDPG-GWO agent with optimized hyperparameters and the original DDPG. In the 2DRobot environment, the original DDPG had rewards ranging from −150 to −50, whereas, in the proposed DDPG-GWO, they ranged from −100 to 100 with a running average between 1 and 800 across 892 episodes. In the MountainCarContinuous environment, the original DDPG struggled with negative rewards, while the proposed DDPG-GWO achieved rewards between 20 and 80 over 218 episodes with a total of 490 timesteps.