Improving Performance of Heterogeneous MapReduce Clusters with Adaptive Task Tuning

• Published in: IEEE transactions on parallel and distributed systems Vol. 28; no. 3; pp. 774 - 786
• Main Authors: Cheng, Dazhao, Rao, Jia, Guo, Yanfei, Jiang, Changjun, Zhou, Xiaobo
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive algorithms; Algorithms; Cloud computing; Cluster analysis; Clustering; Clustering algorithms; Completion time; Configuration management; Configurations; Data centers; genetic algorithm; Genetic algorithms; Hardware; Heterogeneity; heterogeneous clusters; Industries; Interference; MapReduce performance improvement; Nodes; Optimization; Performance degradation; Production scheduling; Self tuning; self-adaptive task tuning; Task scheduling; Tuning; Vector quantization

Datacenter-scale clusters are evolving toward heterogeneous hardware architectures due to continuous server replacement. Meanwhile, datacenters are commonly shared by many users for quite different uses. It often exhibits significant performance heterogeneity due to multi-tenant interferences. The d...