Saarthi: An End-to-End Intelligent Platform for Optimising Distributed Serverless Workloads

TL;DR

Saarthi is an innovative serverless platform that dynamically manages resources and orchestrates functions based on input characteristics, significantly improving throughput and reducing costs compared to static configurations.

Contribution

It introduces an input-aware, end-to-end serverless framework with proactive fault tolerance and multi-objective optimization for resource management.

Findings

Achieves up to 1.45x better throughput

Reduces operational costs by 1.84x

Maintains 98.3% service level targets

Abstract

FaaS offers significant advantages with its infrastructure abstraction, on-demand execution, and attractive no idle resource pricing for modern cloud applications. Despite these benefits, challenges such as startup latencies, static configurations, sub-optimal resource allocation and scheduling still exist due to coupled resource offering and workload-agnostic generic scheduling behaviour. These issues often lead to inconsistent function performance and unexpected operational costs for users and service providers. This paper introduces Saarthi, a novel, end-to-end serverless framework that intelligently manages the dynamic resource needs of function workloads, representing a significant step toward self-driving serverless platforms. Unlike platforms that rely on static resource configurations, Saarthi is input-aware, allowing it to intelligently anticipate resource requirements based on the characteristics of an incoming request payload. This input-driven approach reinforces function right-sizing and enables smart request orchestration across available function configurations. Saarthi further integrates a proactive fault-tolerant redundancy mechanism and employs a multi-objective Integer Linear Programming (ILP) model to maintain an optimal function quantity. This optimisation aims to maximise system throughput while simultaneously reducing overall operational costs. We validate the effectiveness of Saarthi by implementing it as a framework atop OpenFaaS. Our results demonstrate Saarthi's ability to achieve up to 1.45x better throughput, 1.84x reduced costs, while maintaining up to 98.3% service level targets with an overhead of up to 0.2 seconds as compared to the baseline OpenFaaS.