Practical Reinforcement Learning for Adaptive Photolithography Scheduler in Mass Production

• Published in: IEEE transactions on semiconductor manufacturing Vol. 37; no. 1; pp. 16 - 26
• Main Authors: Kim, Eungjin, Kim, Taehyung, Lee, Dongcheol, Kim, Hyeongook, Kim, Sehwan, Kim, Jaewon, Kim, Woosub, Kim, Eunzi, Jin, Younggil, Lee, Tae-Eog
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Active matrix light emitting diode display; Active matrix organic light emitting diodes; Algorithms; Backplanes; Discrete event systems; Fabrication; Job shop scheduling; Lithography; Machine learning; Manufacturing; Mass production; Organic light emitting diodes; Photolithography; Production; Production scheduling; reinforcement learning; Scheduling
• ISSN: 0894-6507; 1558-2345
• DOI: 10.1109/TSM.2023.3336909

This work introduces a practical reinforcement learning (RL) techniques to address the complex scheduling challenges in producing Active Matrix Organic Light Emitting Diode displays. Specifically, we focus on autonomous optimization of the photolithography process, a critical bottleneck in the fabrication. This provides an outperforming scheduling method compared with the existing rule-based approach which requires diverse rules and engineer experience on adapting dynamic environments. Our purposing RL network was designed to make effective schedules aligning with layered structures of the planning and scheduling modules for mass production. In the training phase, historical production data is utilized to create a representative discrete event simulation environment. The RL agent, based on the Deep Q-Network, undergoes episodic training to learn optimal scheduling policies. To ensure safe and reliable scheduling decisions, we further introduce action filters and parallel competing schedulers. The performance of RL-based Scheduler (RLS) is compared to the Rule-Based Scheduler (RBS) over actual fabrication in a year-long period. Based on key performance indicators, we validate the RLS outperforms the RBS, with a remarkable improvement in step target matching, reduced setup times, and enhanced lot assignments. This work also paves a way for the gradual integration of AI-based algorithms into smart manufacturing practices.