ReactiveFnJ: A choreographed model for Fork-Join Workflow in Serverless Computing

• Published in: Journal of cloud computing : advances, systems and applications Vol. 12; no. 1; pp. 63 - 16
• Main Authors: Bharti, Urmil, Goel, Anita, Gupta, S. C.
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2023; Springer Nature B.V; SpringerOpen
• Subjects: Algorithms; Choreography; Cloud computing; Composition; Computer Communication Networks; Computer Science; Computer System Implementation; Computer Systems Organization and Communication Networks; Data processing; Distributed computing; Employment; Event-driven function composition; FaaS; Industrial development; Information Systems Applications (incl.Internet); Linear algebra; Parallel computing; Pipelining (computers); Serverless computing; Software Engineering/Programming and Operating Systems; Special Purpose and Application-Based Systems; Statelessness; Workflow; Orchestration; Event-driven function composition; Parallel computing; Serverless computing; Choreography; FaaS; Distributed computing; Fork and Join
• ISSN: 2192-113X; 2192-113X
• DOI: 10.1186/s13677-023-00429-3

Function-as-a-Service (FaaS) is an event-based reactive programming model where functions run in ephemeral stateless containers for short duration. For building complex serverless applications, function composition is crucial to coordinate and synchronize the workflow of an application. Some serverless orchestration systems exist, but they are in their primitive state and do not provide inherent support for non-trivial workflows like, Fork-Join. To address this gap, we propose a fully serverless and scalable design model ReactiveFnJ for Fork-Join workflow. The intent of this work is to illustrate a design which is completely choreographed, reactive, asynchronous, and represents a dynamic composition model for serverless applications based on Fork-Join workflow. Our design uses two innovative patterns, namely, Relay Composition and Master-Worker Composition to solve execution time-out challenges. As a Proof-of-Concept (PoC), the prototypical implementation of Split-Sort-Merge use case, based on Fork-Join workflow is discussed and evaluated. The ReactiveFnJ handles embarrassingly parallel computations, and its design does not depend on any external orchestration services, messaging services, and queue services. ReactiveFnJ facilitates in designing fully automated pipelines for distributed data processing systems, satisfying the Serverless Trilemma in true essence. A file of any size can be processed using our effective and extensible design without facing execution time-out challenges. The proposed model is generic and can be applied to a wide range of serverless applications that are based on the Fork-Join workflow pattern. It fosters the choreographed serverless composition for complex workflows. The proposed design model is useful for software engineers and developers in industry and commercial organizations, total solution vendors and academic researchers.