Weighted locality-sensitive scheduling for mitigating noise on multi-core clusters

Recent studies have shown that operating system (OS) interference, popularly called OS noise can be a significant problem as we scale to a large number of processors. One solution for mitigating noise is to turn off certain OS services on the machine. However, this is typically infeasible because fu...

Full description

Saved in:
Bibliographic Details
Published in2011 18th International Conference on High Performance Computing pp. 1 - 10
Main Authors Kale, V., Bhatele, A., Gropp, W. D.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.12.2011
Subjects
Dynamic scheduling
Load management
Noise
Operating systems
Processor scheduling
Program processors
Tuning
Online AccessGet full text

Cover

More Information
Summary:Recent studies have shown that operating system (OS) interference, popularly called OS noise can be a significant problem as we scale to a large number of processors. One solution for mitigating noise is to turn off certain OS services on the machine. However, this is typically infeasible because full-scale OS services may be required for some applications. Furthermore, it is not a choice that an end user can make. Thus, we need an application-level solution. Building upon previous work that demonstrated the utility of within-node light-weight load balancing, we discuss the technique of weighted micro-scheduling and provide insights based on experimentation for two different machines with very different noise signatures. Through careful enumeration of the search space of scheduler parameters, we allow our weighted micro-scheduler to be dynamic, adaptive and tunable for a specific application running on a specific architecture. By doing this, we show how we can enable running scientific applications efficiently on a very large number of processors, even in the presence of noise.
ISBN:1457719517
9781457719516
ISSN:1094-7256
2640-0316
DOI:10.1109/HiPC.2011.6152722