Multiple job co-allocation strategy for heterogeneous multi-cluster systems based on linear programming

• Published in: Journal of supercomputing Vol. 58; no. 3; pp. 394 - 402
• Main Authors: Blanco, Héctor, Lérida, Josep Lluís, Cores, Fernando, Guirado, Fernando
• Format: Journal Article Conference Proceeding
• Language: English
• Published: Boston Springer US 01.12.2011; Springer
• Subjects: Applied sciences; Compilers; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Exact sciences and technology; Interpreters; Operational research and scientific management; Operational research. Management science; Processor Architectures; Programming Languages; Scheduling, sequencing; Software; Multi-cluster systems; Mixed integer programming; Co-allocation; Job scheduling; Resource allocation; Task scheduling; Calculator cluster; Linear programming; Scheduling; Distributed system; Distributed computing; Selection function; Time response; Linear model; Workload; Heterogeneity; Response time; Parallelism; Queue; Execution time
• ISSN: 0920-8542; 1573-0484
• DOI: 10.1007/s11227-011-0596-2

Multi-cluster environments are composed of multiple clusters of computers that act collaboratively, and thus allowing computational problems to be treated that require more resources than those available in a single cluster. However, the degree of complexity of the scheduling process is greatly increased by the heterogeneity of resources and co-allocation process, which distributes the tasks of parallel jobs across cluster boundaries. This work presents a new scheduling strategy that allocates multiple jobs from the system queue simultaneously on a heterogeneous multicluster, by applying co-allocation when is necessary. Our strategy is composed by a job selection function and a linear programming model to find the best allocation for multiple jobs. The proposed scheduling technique is shown to reduce the execution times of the parallel jobs and the overall response times by 38% compared with other scheduling techniques in the literature.