Adaptive service rate control of an M/M/1 queue with server breakdowns

• Published in: Queueing systems Vol. 106; no. 1-2; pp. 159 - 191
• Main Authors: Zheng, Yi, Julaiti, Juxihong, Pang, Guodong
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2024; Springer Nature B.V
• Subjects: Adaptive algorithms; Adaptive control; Breakdown; Business and Management; Computer Communication Networks; Control; Control algorithms; Control theory; Ergodic processes; Finite capacity; Markov processes; Operations Research/Decision Theory; Optimization; Policies; Probability Theory and Stochastic Processes; Queuing theory; Self tuning; Supply Chain Management; Systems Theory; Service rate control; 90C40; 93E20; Finite or infinite capacity; Server breakdowns; Adaptive (self-tuning) control; 93E35; Discounted and long-run average (ergodic) cost criteria; 1 queue; Markov decision process; 60K25; /
• ISSN: 0257-0130; 1572-9443
• DOI: 10.1007/s11134-023-09900-z

We study service rate control problems for an M / M /1 queue with server breakdowns in which the breakdown rate is assumed to be a function of the service rate. Assuming that the queue has infinite capacity, we first establish the optimality equations for the discounted cost problem and characterize the optimal rate control policies. Then, we characterize the ergodicity of the controlled queue and establish the optimality conditions for the average-cost (ergodic) control problem using the vanishing discounted method. We next study the ergodic control problem when the queue has a finite capacity and establish a verification theorem by directly involving the stationary distribution of the controlled Markov process. For practical applications, we consider the adaptive service rate control problem for the model with finite capacity. Studying this problem is useful because the relationship between the server breakdown rate and the service rate is costly to observe in practice. We propose an adaptive (self-tuning) control algorithm, assuming that the relationship between the server breakdown rate and the service rate is linear with unknown parameters. We prove that the regret vanishes under the algorithm and the proposed policies are asymptotically optimal. In addition, numerical experiments are conducted to validate the algorithm.