Deep Reinforcement Learning for Optimal Replenishment in Stochastic Assembly Systems

• Published in: Mathematics (Basel) Vol. 13; no. 14; p. 2229
• Main Authors: Sid Ahmed Abdellahi, Lativa, Zoubeir, Zeinebou, Mohamed, Yahya, Haouba, Ahmedou, Hmetty, Sidi
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.07.2025
• Subjects: Algorithms; Artificial intelligence; assembly system; Convergence (Social sciences); Costs; Customer satisfaction; Decision making; Deep learning; deep reinforcement learning; Demand; Design; Fines & penalties; Inventory; Inventory control; Inventory management; Lead time; Linear programming; Logistics; Manufacturing; Markov analysis; Markov processes; Modular structures; Neural networks; Optimization; Order quantity; Policies; Production planning; Random variables; Randomness; Raw materials; Replenishment; replenishment planning; Shortages; stochastic demand; Suppliers; Supply chains; uncertain lead times; Germany
• ISSN: 2227-7390; 2227-7390
• DOI: 10.3390/math13142229

This study presents a reinforcement learning–based approach to optimize replenishment policies in the presence of uncertainty, with the objective of minimizing total costs, including inventory holding, shortage, and ordering costs. The focus is on single-level assembly systems, where both component delivery lead times and finished product demand are subject to randomness. The problem is formulated as a Markov decision process (MDP), in which an agent determines optimal order quantities for each component by accounting for stochastic lead times and demand variability. The Deep Q-Network (DQN) algorithm is adapted and employed to learn optimal replenishment policies over a fixed planning horizon. To enhance learning performance, we develop a tailored simulation environment that captures multi-component interactions, random lead times, and variable demand, along with a modular and realistic cost structure. The environment enables dynamic state transitions, lead time sampling, and flexible order reception modeling, providing a high-fidelity training ground for the agent. To further improve convergence and policy quality, we incorporate local search mechanisms and multiple action space discretizations per component. Simulation results show that the proposed method converges to stable ordering policies after approximately 100 episodes. The agent achieves an average service level of 96.93%, and stockout events are reduced by over 100% relative to early training phases. The system maintains component inventories within operationally feasible ranges, and cost components—holding, shortage, and ordering—are consistently minimized across 500 training episodes. These findings highlight the potential of deep reinforcement learning as a data-driven and adaptive approach to inventory management in complex and uncertain supply chains.