KubeAdaptor: A Docking Framework for Workflow Containerization on Kubernetes

As Kubernetes becomes the infrastructure of the cloud-native era, the integration of workflow systems with Kubernetes is gaining more and more popularity. To our knowledge, workflow systems employ scheduling algorithms that optimize task execution order of workflow to improve performance and executi...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Shan, Chenggang, Wang, Guan, Xia, Yuanqing, Zhan, Yufeng, Zhang, Jinhui
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 04.07.2022
Subjects
Algorithms
Consistency
Docking
Performance enhancement
Scheduling
Task scheduling
Workflow management systems
Online AccessGet full text

Cover

More Information
Summary:As Kubernetes becomes the infrastructure of the cloud-native era, the integration of workflow systems with Kubernetes is gaining more and more popularity. To our knowledge, workflow systems employ scheduling algorithms that optimize task execution order of workflow to improve performance and execution efficiency. However, due to its inherent scheduling mechanism, Kubernetes does not execute containerized scheduling following the optimized task execution order of workflow amid migrating workflow systems to the Kubernetes platform. This inconsistency in task scheduling order seriously degrades the efficiency of workflow execution and brings numerous challenges to the containerized process of workflow systems on Kubernetes. In this paper, we propose a cloud-native workflow engine, also known as KubeAdaptor, a docking framework able to implement workflow containerization on Kubernetes, integrate workflow systems with Kubernetes, ensuring the consistency of task scheduling order. We introduce the design and architecture of the KubeAdaptor, elaborate on the functionality implementation and the event-trigger mechanism within the KubeAdaptor. Experimental results about four real-world workflows show that the KubeAdaptor ensures the consistency of the workflow systems and Kubernetes in the task scheduling order. Compared with the baseline Argo workflow engine, the KubeAdaptor achieves better performance in terms of the average execution time of task pod, average workflow lifecycle, and resource usage rate.
ISSN:2331-8422