Distributed Double Machine Learning with a Serverless Architecture

TL;DR

This paper presents a serverless architecture for double machine learning, enabling fast, cost-effective, and parallelizable estimations using AWS Lambda, with a practical Python prototype and case study.

Contribution

It introduces a novel serverless implementation of double machine learning, leveraging cloud parallelism to improve efficiency and ease of deployment.

Findings

Significant reduction in estimation time

Cost savings through serverless architecture

Effective parallelization of double machine learning tasks

Abstract

This paper explores serverless cloud computing for double machine learning. Being based on repeated cross-fitting, double machine learning is particularly well suited to exploit the high level of parallelism achievable with serverless computing. It allows to get fast on-demand estimations without additional cloud maintenance effort. We provide a prototype Python implementation \texttt{DoubleML-Serverless} for the estimation of double machine learning models with the serverless computing platform AWS Lambda and demonstrate its utility with a case study analyzing estimation times and costs.