Evaluating Serverless Machine Learning Performance on Google Cloud Run

TL;DR

This paper evaluates how well Google Cloud Run, a serverless platform not specifically designed for machine learning, handles machine learning workloads, focusing on performance aspects.

Contribution

It provides an empirical analysis of machine learning performance on Google Cloud Run, highlighting challenges and considerations for deploying ML services on such platforms.

Findings

Performance metrics indicate limitations for ML workloads

Cost and scalability trade-offs are identified

Insights into platform suitability for ML applications

Abstract

End-users can get functions-as-a-service from serverless platforms, which promise lower hosting costs, high availability, fault tolerance, and dynamic flexibility for hosting individual functions known as microservices. Machine learning tools are seen to be reliably useful, and the services created using these tools are in increasing demand on a large scale. The serverless platforms are uniquely suited for hosting these machine learning services to be used for large-scale applications. These platforms are well known for their cost efficiency, fault tolerance, resource scaling, robust APIs for communication, and global reach. However, machine learning services are different from the web-services in that these serverless platforms were originally designed to host web services. We aimed to understand how these serverless platforms handle machine learning workloads with our study. We examine machine learning performance on one of the serverless platforms - Google Cloud Run, which is a GPU-less infrastructure that is not designed for machine learning application deployment.