Joint Workload Distribution and Capacity Augmentation in Hybrid Datacenter Networks

In hybrid datacenter networks, wired connections are augmented with wireless links to facilitate data transfers between racks. The usage of mmWave/FSO wireless links enables dynamic bandwidth/capacity allocation with extremely small reconfiguration delay. Also, on-demand workload distribution, where...

Full description

Saved in:
Bibliographic Details
Published inIEEE/ACM transactions on networking Vol. 29; no. 1; pp. 120 - 133
Main Authors Ao, Weng Chon, Huang, Po-Han, Psounis, Konstantinos
Format Journal Article
LanguageEnglish
Published New York IEEE 01.02.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Algorithms
Augmentation
biconvex optimization
Completion time
Data centers
distributed and parallel computing
hybrid datacenter networks
Millimeter waves
On-demand workload distribution
Optimization
Pipeline processing
Pipelining (computers)
Racks
Reconfiguration
Resource management
Servers
Task analysis
wireless capacity augmentation
Wireless communication
Workload
Workloads
Online AccessGet full text

Cover

More Information
Summary:In hybrid datacenter networks, wired connections are augmented with wireless links to facilitate data transfers between racks. The usage of mmWave/FSO wireless links enables dynamic bandwidth/capacity allocation with extremely small reconfiguration delay. Also, on-demand workload distribution, where the workload of a job is divided into multiple tasks that can be distributed/routed to different racks to be processed in parallel, allows better utilization of computational resources in data centers. In prior work, the dynamic wireless capacity augmentation and workload distribution decisions were mostly made independently and in a heuristic manner for serving distributed and parallel computing jobs. In this paper, we propose a novel analytical framework and algorithms to jointly optimize both the wireless capacity augmentation and the workload distribution, to minimize the job completion time. We consider workload that is not amenable to pipelining, fully amenable to pipelining, and partially amenable to pipelining. With extensive simulation studies, we show that the gain (in terms of the reduction in the job completion time) can be very substantial when allowing such joint optimization.
ISSN:1063-6692
1558-2566
DOI:10.1109/TNET.2020.3027607