Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines

• Published in: Big Data Analytics and Knowledge Discovery pp. 133 - 148
• Main Authors: Abbas, Ammar N., Chasparis, Georgios C., Kelleher, John D.
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep Reinforcement Learning (DRL); Input-Output Hidden Markov Model (IOHMM); Interpretable AI; Predictive maintenance
• ISBN: 9783031126697; 3031126696
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-031-12670-3_12

An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes on combining the advantages of input-output hidden Markov models and reinforcement learning. We propose a novel hierarchical modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while at a low level, provides the optimal replacement policy. This approach outperforms baseline deep reinforcement learning (DRL) models and has performance comparable to that of a state-of-the-art reinforcement learning system while being more interpretable.