RBAY: A Scalable and Extensible Information Plane for Federating Distributed Datacenter Resources

• Published in: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) pp. 1345 - 1355
• Main Authors: Xin Chen, Liting Hu, Blough, Douglas M., Kozuch, Michael A., Wolf, Matthew
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2017
• Subjects: active attribute; admin-customized handler; Aggregates; Cloud computing; Graphics processing units; hierarchical aggregation tree; Monitoring; Peer-to-peer computing; RBay; Routing; Scalability
• ISSN: 1063-6927; 2575-8411
• DOI: 10.1109/ICDCS.2017.42

While many institutions, whether industrial, academic, or governmental, satisfy their computing needs through public cloud providers, many others still manage their own resources, often as geographically distributed datacenters. Spare capacity from these geographically distributed datacenters could be offered to others, provided there were a mechanism to discover, and then request these resources. Unfortunately, single datacenter administrators tend not to cooperate due to issues of scalability, diverse administrative policies, and site-specific monitoring infrastructure. This paper describes RBAY, an integrated information plane that enables secure and scalable sharing between geographically distributed datacenters. RBAY's key design features are twofold. First, RBAY employs a decentralized `hierarchical aggregation tree' structure to seamlessly aggregate spare resources from geographically distributed datacenters to a global information plane. Second, RBAY attaches to each participating server a `admin-customized' handler, which follows site-specific policy to expose, hide, add, remove resources to RBAY, and thus fulfill the task of `which resource to expose to whom, when, and how'. An experimental evaluation on eight real-world geo-distributed sites demonstrates RBAY's rapid response to composite queries, as well as its extensible, scalable, and lightweight nature.