An Adaptive Resource Sharing Platform for Internet Data Center

• Published in: 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing pp. 1179 - 1186
• Main Authors: Qing Li, Yong Li, Bibo Tu, Dan Meng
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2013
• Subjects: Arrays; Games; internet data center; job-resource description protocol; job-resource matching algorithm; Protocols; Resource management; resource utilization; scalability; Servers; Web services
• DOI: 10.1109/HPCC.and.EUC.2013.167

Internet applications are diverse, such as online web services, offline data analysis jobs and so on. To make different types of applications work effectively, many programming models and computing frameworks are designed, such as MapReduce, Dryad, and Hadoop. How to support the emerging applications and computing frameworks with high resource utilization is a serious challenge. In this paper, we present a resource sharing platform which can support different types of applications and computing frameworks by adopting a job-resource description protocol. To improve the resource utilization without impairing the performances of applications, we propose a job-resource matching algorithm, called JRMatching, which deploys those different types of jobs on one node hybridly to reduce resource fragments, and a mechanism to provide resources adaptively, which can reduce resource waste caused by fixed resource allocation. The experiments show that the performances of applications sharing resources in one node are close to running exclusively and the resource utilization of the internet data center also can be improved.