PanOpticon: A Comprehensive Benchmarking Tool for Serverless Applications

• Published in: 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS) pp. 144 - 151
• Main Authors: Somu, Nikhila, Daw, Nilanjan, Bellur, Umesh, Kulkarni, Purushottam
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2020
• Subjects: automation; Benchmark testing; benchmarking; Computer architecture; cross-platform; FAA; Google; Measurement; serverless computing; Throughput
• ISSN: 2155-2509
• DOI: 10.1109/COMSNETS48256.2020.9027346

Serverless computing manifests as the FaaS offering where clients submit code to be managed and run by the service provider in the place of hiring and managing VMs for this purpose. Multiple Cloud Service providers have come up with their own implementations of FaaS infrastructure providing end-users with a multitude of choices. Each such platform provides a non-overlapping set of features which satisfies a subset of users. Further the design of the platform dictates the performance overheads of triggering the function. A tool that automates capturing how a function behaves under different configurations of a platform and across platforms will, therefore, be useful for end-users intending to deploy applications as a collection of FaaS units. In spite of the presence of a few benchmarking tools for FaaS offerings, they lack the comprehensive breadth required to understand the performance aspects of the design choices made by the end-users. Most tools focus on tuning resource parameters like memory, CPU requirements and measure metrics like execution time. They lack the option to measure the effects of features such as function chaining and choice of function triggers. We present PanOpticon - a tool that automates the deployment of FaaS applications on different platforms under a set of tunable configuration choices and presents the users with performance measurements for each configuration and platform.