Challenges in Deploying Machine Learning: a Survey of Case Studies

TL;DR

This survey reviews real-world case studies of deploying machine learning models, highlighting challenges at each deployment stage and proposing a research agenda to address these issues.

Contribution

It systematically maps deployment challenges across different industries and stages, providing a comprehensive overview and identifying areas for future research.

Findings

Challenges exist at every stage of deployment

Practitioners face issues in data collection, model training, and maintenance

A research agenda is proposed to address deployment hurdles

Abstract

In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process. The goal of this paper is to lay out a research agenda to explore approaches addressing these challenges.