Cost-Efficient Scheduling of Bulk Transfers in Inter-Datacenter WANs

• Published in: IEEE/ACM transactions on networking Vol. 27; no. 5; pp. 1973 - 1986
• Main Authors: Yang, Zhenjie, Cui, Yong, Wang, Xin, Liu, Yadong, Li, Minming, Xiao, Shihan, Li, Chuming
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Approximation algorithms; Bandwidth; Bandwidths; Bottlenecks; Computer simulation; cost; Cost engineering; Data centers; Data transfer (computers); Deadlines; Handheld computers; IEEE transactions; Inter-datacenter WAN; Schedules; Scheduling; Wide area networks
• ISSN: 1063-6692; 1558-2566
• DOI: 10.1109/TNET.2019.2934896

With the quick growth of traffic between data centers, inefficient transfer scheduling in inter-datacenter networks can lead to a huge waste of bandwidth thus significant bandwidth cost. Previous work have explored different ways, such as software-defined WANs and dynamic pricing mechanisms, to overcome the inefficiency of inter-datacenter networks. However, there is a big challenge in addressing the fundamental conflicts between the deadline-aware transfer scheduling and minimizing the bandwidth cost. Unlike existing efforts that schedule inter-datacenter transfers under fixed link capacities, wherein some deadlines are violated and the service quality is degraded, we aim to finish all the transfers on time with as little bandwidth as possible to minimize the bandwidth cost. We take into account the variation of bandwidth price and the deadline requirements of services, and formulate the problem of cost-efficient scheduling of bulk transfers with deadline guarantee, which is shown to be NP-hard. Benefitting from the relax-and-round method, we propose a progressively-descending algorithm (PDA) to schedule bulk transfers and meet the above goals with a guaranteed approximation ratio. We apply our algorithm in a bulk transfer scheduler, Butler, and build a small-scale testbed to evaluate its efficiency. Both large-scale simulation and testbed experiment results validate the ability of our scheme on cutting down the bandwidth cost. Compared with existing approaches, it reduces up to 60% bandwidth cost and increases the network utilization by up to 140%.