Applying a Requirements-Focused Agile Management Approach for Machine Learning-Enabled Systems

TL;DR

This paper presents RefineML, a requirements-focused agile approach tailored for ML-enabled systems, demonstrating high acceptance and practical benefits in industry-academia collaboration, while addressing unique challenges of ML integration.

Contribution

It introduces RefineML, a novel approach integrating ML-specific requirements with agile practices, and provides empirical evidence of its effectiveness in real-world projects.

Findings

High perceived usefulness and acceptance among stakeholders

Improved communication and early feasibility assessments

Enables dual-track governance for ML and software development

Abstract

Machine Learning (ML)-enabled systems challenge traditional Requirements Engineering (RE) and agile management due to data dependence, experimentation, and uncertain model behavior. Existing RE and agile practices remain poorly integrated and insufficiently tailored to these characteristics. This paper reports on the practical experience of applying RefineML, a requirements-focused approach for the continuous and agile refinement of ML-enabled systems, which integrates ML-tailored specification and agile management approaches with best practices derived from a systematic mapping study. The application context concerns an industry-academia collaboration project between PUC-Rio and EXA, a Brazilian cybersecurity company. For evaluation purposes, we applied questionnaires assessing RefineML's suitability and overall acceptance and semi-structured interviews. We applied thematic analysis to the collected qualitative data. Regarding suitability and acceptance, the results of the questionnaires indicated high perceived usefulness and intention to use. Based on the interviews, stakeholders perceived RefineML as improving communication and facilitating early feasibility assessments, as well as enabling dual-track governance of ML and software work, allowing continuous refinement of the model while evolving the overall software project. However, some limitations remain, particularly related to difficulties in operationalizing ML concerns into agile requirements and in estimating ML effort.