A Catalogue of Concerns for Specifying Machine Learning-Enabled Systems

TL;DR

This paper presents a validated catalogue of 45 concerns across five perspectives to improve requirements engineering for ML-enabled systems, aiming to enhance their reliability and development process.

Contribution

It introduces a comprehensive, validated set of concerns for specifying ML-enabled systems, based on industrial insights and expert validation.

Findings

The catalogue covers objectives, user experience, infrastructure, model, and data perspectives.

Focus group feedback confirmed the catalogue's practical relevance and usability.

The set of concerns supports requirements engineers in developing more reliable ML systems.

Abstract

Requirements engineering (RE) activities for machine learning (ML) are not well-established and researched in the literature. Many issues and challenges exist when specifying, designing, and developing ML-enabled systems. Adding more focus on RE for ML can help to develop more reliable ML-enabled systems. Based on insights collected from previous work and industrial experiences, we propose a catalogue of 45 concerns to be considered when specifying ML-enabled systems, covering five different perspectives we identified as relevant for such systems: objectives, user experience, infrastructure, model, and data. Examples of such concerns include the execution engine and telemetry for the infrastructure perspective, and explainability and reproducibility for the model perspective. We conducted a focus group session with eight software professionals with experience developing ML-enabled systems to validate the importance, quality and feasibility of using our catalogue. The feedback allowed us to improve the catalogue and confirmed its practical relevance. The main research contribution of this work consists in providing a validated set of concerns grouped into perspectives that can be used by requirements engineers to support the specification of ML-enabled systems.