Online optimization scheduling for scientific workflows with deadline constraint on hybrid clouds

• Published in: Concurrency and computation Vol. 28; no. 11; pp. 3079 - 3095
• Main Authors: Lin, Bing, Guo, Wenzhong, Lin, Xiuyan
• Format: Journal Article
• Language: English
• Published: Blackwell Publishing Ltd 10.08.2016
• Subjects: Algorithms; Clouds; Computation; deadline constraint; hybrid clouds; Markets; online optimization scheduling; Partitions; Scheduling; scientific workflows; Strategy; Workflow
• ISSN: 1532-0626; 1532-0634
• DOI: 10.1002/cpe.3582

Summary The tremendous parallel computing ability of cloud computing encourages investigators to research its drawbacks and advantages on processing large‐scale scientific applications such as workflows. The current cloud market is composed of numerous diverse public clouds and a local private cloud, and workflow scheduling is one of the biggest challenges on hybrid clouds due to the highly fragmented cloud market with respect to service provisions, pricing models, and bandwidths. In this paper, we propose an online‐scheduling strategy for continuous submitted scientific workflows on hybrid clouds, which aims to complete the deadline‐constrained applications as more as possible at a lower price. Firstly, a hierarchical iterative application partition (HIAP) algorithm is proposed to partition the application into a set of dependent tasks. Moreover, many online‐scheduling algorithms cooperated with HIAP are presented to finish the workflows with a low average payment. Our strategy takes into account the basic characteristics on hybrid clouds such as bandwidth constraints, data transfer cost and computational cost. Various well‐known workflows are used for evaluating the multiple scheduling algorithms and the results show that the MLF_ID approach can achieve a promising performance. Copyright © 2015 John Wiley & Sons, Ltd.