On Optimizing Operational Efficiency in Storage Systems via Deep Reinforcement Learning

• Published in: Machine Learning and Knowledge Discovery in Databases Vol. 11053; pp. 238 - 253
• Main Authors: Srinivasa, Sunil, Kathalagiri, Girish, Varanasi, Julu Subramanyam, Quintela, Luis Carlos, Charafeddine, Mohamad, Lee, Chi-Hoon
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2019; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Actor-critic methods; Data center; Deep reinforcement learning; Operational efficiency; Storage system
• ISBN: 9783030109967; 3030109968
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-10997-4_15

This paper deals with the application of deep reinforcement learning to optimize the operational efficiency of a solid state storage rack. Specifically, we train an on-policy and model-free policy gradient algorithm called the Advantage Actor-Critic (A2C). We deploy a dueling deep network architecture to extract features from the sensor readings off the rack and devise a novel utility function that is used to control the A2C algorithm. Experiments show performance gains greater than 30% over the default policy for deterministic as well as random data workloads.