I/O-signature-based feature analysis and classification of high-performance computing applications

• Published in: Cluster computing Vol. 27; no. 3; pp. 3219 - 3231
• Main Authors: Park, Ju-Won, Huang, Xin, Lee, Jae-Kook, Hong, Taeyoung
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2024
• Subjects: Computer Communication Networks; Computer Science; Operating Systems; Processor Architectures; High performance computing; Key features; I/O patterns analysis
• ISSN: 1386-7857; 1573-7543
• DOI: 10.1007/s10586-023-04139-y

The demand for high-performance computing (HPC) resources in computing fields such as machine learning has increased significantly in recent years. Computing power has been growing exponentially to keep up with this demand. However, these gains have not been able to translate to performance improvement in real-world applications. One of the biggest reasons for this is performance degradation in terms of input/output (I/O) due to the increased storage latency and complex parallel I/O architecture of accessing data in distributed storage systems. In this study, we analyze application-specific I/O patterns to gain a deeper understanding of I/O throughput and the interaction between applications and the I/O system. Specifically, we analyze the importance of each feature of I/O patterns through feature analysis based on the collected monitoring information. We also investigate the feasibility of identifying the application based on the analyzed key features. To this end, we present the analysis accuracy and confusion matrix of four algorithms, including random forest, which are widely used as classification algorithms in the experimental results. The experiment results confirm that we can distinguish applications with an accuracy of more than 90% by using application-specific I/O patterns.