MLASP: Machine learning assisted capacity planning An industrial experience report

• Published in: Empirical software engineering : an international journal Vol. 26; no. 5
• Main Authors: Vitui, Arthur, Chen, Tse-Hsun (Peter)
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2021
• Subjects: Compilers; Computer Science; Experience Report; Interpreters; Programming Languages; Software Engineering/Programming and Operating Systems; Capacity testing; Deep learning; Performance testing; Load testing; Machine learning
• ISSN: 1382-3256; 1573-7616
• DOI: 10.1007/s10664-021-09994-0

In industrial environments it is critical to find out the capacity of a system and plan for a deployment layout that meets the production traffic demands. The system capacity is influenced by both the performance of the system’s constituting components and the physical environment setup. In a large system, the configuration parameters of individual components give the flexibility to developers and load test engineers to tune system performance without changing the source code. However, due to the large search space, estimating the capacity of the system given different configuration values is a challenging and costly process. In this paper, we propose an approach, called MLASP , that uses machine learning models to predict the system key performance indicators (i.e., KPIs), such as throughput, given a set of features made off configuration parameter values, including server cluster setup, to help engineers in capacity planning for production environments. Under the same load, we evaluate MLASP on two large-scale mission-critical enterprise systems developed by Ericsson and on one open-source system. We find that: 1) MLASP can predict the system throughput with a very high accuracy. The difference between the predicted and the actual throughput is less than 1%; and 2) By using only a small subset of the training data (e.g., 3% of the entire data for the open-source system), MLASP can still predict the throughput accurately. We also document our experience of successfully integrating the approach into an industrial setting. In summary, this paper highlights the benefits and potential of using machine learning models to assist load test engineers in capacity planning.