FaaSCtrl: A Comprehensive-Latency Controller for Serverless Platforms

• Published in: IEEE transactions on cloud computing Vol. 12; no. 4; pp. 1328 - 1343
• Main Authors: Panda, Abhisek, Sarangi, Smruti R.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Complexity; Degradation; Design optimization; Design standards; Distributed processing; Job colocation; Knobs; Measurement; Optimization; performance interference; Performance measurement; Priority scheduling; Quality of service; Real time; Real-time systems; Reinforcement learning; resource scheduling; Schedules; Serverless computing; Tail
• DOI: 10.1109/TCC.2024.3473015
• ISSN: 2168-7161

Serverless computing systems have become very popular because of their natural advantages with respect to auto-scaling, load balancing and fast distributed processing. As of today, almost all serverless systems define two QoS classes: best-effort (<inline-formula><tex-math notation="LaTeX">BE</tex-math> <mml:math><mml:mrow><mml:mi>B</mml:mi><mml:mi>E</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq1-3473015.gif"/> </inline-formula>) and latency-sensitive (<inline-formula><tex-math notation="LaTeX">LS</tex-math> <mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>S</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq2-3473015.gif"/> </inline-formula>). Systems typically do not offer any latency or QoS guarantees for <inline-formula><tex-math notation="LaTeX">BE</tex-math> <mml:math><mml:mrow><mml:mi>B</mml:mi><mml:mi>E</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq3-3473015.gif"/> </inline-formula> jobs and run them on a best-effort basis. In contrast, systems strive to minimize the processing time for <inline-formula><tex-math notation="LaTeX">LS</tex-math> <mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>S</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq4-3473015.gif"/> </inline-formula> jobs. This work proposes a precise definition for these job classes and argues that we need to consider a bouquet of performance metrics for serverless applications, not just a single one. We thus propose the comprehensive latency (<inline-formula><tex-math notation="LaTeX">CL</tex-math> <mml:math><mml:mrow><mml:mi>C</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq5-3473015.gif"/> </inline-formula>) that comprises the mean, tail latency, median and standard deviation of a series of invocations for a given serverless function. Next, we design a system FaaSCtrl , whose main objective is to ensure that every component of the <inline-formula><tex-math notation="LaTeX">CL</tex-math> <mml:math><mml:mrow><mml:mi>C</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="panda-ieq6-3473015.gif"/> </inline-formula> is within a prespecified limit for an LS application, and for BE applications, these components are minimized on a best-effort basis. Given the sheer complexity of the scheduling problem in a large multi-application setup, we use the method of surrogate functions in optimization theory to design a simpler optimization problem that relies on performance and fairness. We rigorously establish the relevance of these metrics through characterization studies. Instead of using standard approaches based on optimization theory, we use a much faster reinforcement learning (RL) based approach to tune the knobs that govern process scheduling in Linux, namely the real-time priority and the assigned number of cores. RL works well in this scenario because the benefit of a given optimization is probabilistic in nature, owing to the inherent complexity of the system. We show using rigorous experiments on a set of real-world workloads that FaaSCtrl achieves its objectives for both LS and BE applications and outperforms the state-of-the-art by 36.9% (for tail response latency) and 44.6% (for response latency's std. dev.) for LS applications.