Serverless Data Science -- Are We There Yet? A Case Study of Model Serving

Machine learning (ML) is an important part of modern data science applications. Data scientists today have to manage the end-to-end ML life cycle that includes both model training and model serving, the latter of which is essential, as it makes their works available to end-users. Systems of model se...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Wu, Yuncheng, Tien Tuan Anh Dinh, Hu, Guoyu, Zhang, Meihui, Yeow Meng Chee, Beng Chin Ooi
Format Paper Journal Article
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 01.03.2022
Subjects
Cloud computing
Computer Science - Artificial Intelligence
Computer Science - Databases
Computer Science - Distributed, Parallel, and Cluster Computing
Computer Science - Learning
Data science
End users
Graphics processing units
Life cycle engineering
Machine learning
Performance evaluation
Scientists
User requirements
Web services
Workload
Workloads
Online AccessGet full text

Cover

More Information
Summary:Machine learning (ML) is an important part of modern data science applications. Data scientists today have to manage the end-to-end ML life cycle that includes both model training and model serving, the latter of which is essential, as it makes their works available to end-users. Systems of model serving require high performance, low cost, and ease of management. Cloud providers are already offering model serving choices, including managed services and self-rented servers. Recently, serverless computing, whose advantages include high elasticity and a fine-grained cost model, brings another option for model serving. Our goal in this paper is to examine the viability of serverless as a mainstream model serving platform. To this end, we first conduct a comprehensive evaluation of the performance and cost of serverless against other model serving systems on Amazon Web Service and Google Cloud Platform. We find that serverless outperforms many cloud-based alternatives. Further, there are settings under which it even achieves better performance than GPU-based systems. Next, we present the design space of serverless model serving, which comprises multiple dimensions, including cloud platforms, serving runtimes, and other function-specific parameters. For each dimension, we analyze the impact of different choices and provide suggestions for data scientists to better utilize serverless model serving. Finally, we discuss challenges and opportunities in building a more practical serverless model serving system.
ISSN:2331-8422
DOI:10.48550/arxiv.2103.02958