A case study of a shared/buy-in computing ecosystem

• Published in: Cluster computing Vol. 21; no. 3; pp. 1595 - 1606
• Main Authors: Liao, Christopher, Klausner, Yonatan, Starobinski, David, Simhon, Eran, Bestavros, Azer
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2018; Springer Nature B.V
• Subjects: Algorithms; Archives & records; Case studies; Clusters; Computation; Computer Communication Networks; Computer Science; Employment; Operating Systems; Performance evaluation; Processor Architectures; Quality of service; Research facilities; Run time (computers); Scheduling; Text editing; Workloads; Grid computing; Shared/buy-in architecture; Cluster computing; Workload characterization
• ISSN: 1386-7857; 1573-7543
• DOI: 10.1007/s10586-018-2256-2

Many research institutions are deploying computing clusters based on a shared/buy-in paradigm. Such clusters combine shared computers, which are free to be used by all users, and buy-in computers, which are computers purchased by users for semi-exclusive use. The purpose of this paper is to characterize the typical behavior and performance of a shared/buy-in computing cluster, using data traces from the Shared Computing Cluster (SCC) at Boston University that runs under this paradigm as a case study. Among our main findings, we show that the semi-exclusive policy, which allows any SCC user to use idle buy-in resources for a limited time, increases the utilization of buy-in resources by 17.4%, thus significantly improving the performance of the system as a whole. We find that jobs allowed to run on idle buy-in resources arrive more frequently and run for a shorter time than other jobs. Finally, we identify the run time limit (i.e., the maximum time during which a job is allowed to use resources) and the type of parallel environment as two factors that have a significant impact on the different performance experienced by shared and buy-in jobs.