Agile Management for Machine Learning: A Systematic Mapping Study

TL;DR

This paper systematically reviews how agile management practices are applied to ML-enabled systems, highlighting frameworks, key themes, and challenges like effort estimation, to guide future research and practice.

Contribution

It provides a comprehensive mapping of existing research, frameworks, and key themes in agile management for ML systems, identifying gaps and areas needing empirical validation.

Findings

Identified 27 relevant papers from 2008 to 2024.

Mapped eight frameworks and categorized key themes.

Highlighted effort estimation as a major challenge.

Abstract

[Context] Machine learning (ML)-enabled systems are present in our society, driving significant digital transformations. The dynamic nature of ML development, characterized by experimental cycles and rapid changes in data, poses challenges to traditional project management. Agile methods, with their flexibility and incremental delivery, seem well-suited to address this dynamism. However, it is unclear how to effectively apply these methods in the context of ML-enabled systems, where challenges require tailored approaches. [Goal] Our goal is to outline the state of the art in agile management for ML-enabled systems. [Method] We conducted a systematic mapping study using a hybrid search strategy that combines database searches with backward and forward snowballing iterations. [Results] Our study identified 27 papers published between 2008 and 2024. From these, we identified eight frameworks and categorized recommendations and practices into eight key themes, such as Iteration Flexibility, Innovative ML-specific Artifacts, and the Minimal Viable Model. The main challenge identified across studies was accurate effort estimation for ML-related tasks. [Conclusion] This study contributes by mapping the state of the art and identifying open gaps in the field. While relevant work exists, more robust empirical evaluation is still needed to validate these contributions.