Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach

TL;DR

This paper introduces PerSpecML, a perspective-based approach that helps practitioners identify critical attributes and concerns in specifying ML-enabled systems, improving communication and system quality.

Contribution

The paper presents PerSpecML, a novel approach that analyzes 59 concerns across five perspectives to enhance specification practices for ML-enabled systems.

Findings

Validated in academic and industrial contexts

Helps reveal overlooked components in ML system specifications

Improves communication among interdisciplinary teams

Abstract

Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this paper, we present PerSpecML, a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system's quality. The approach involves analyzing 59 concerns related to typical tasks that practitioners face in ML projects, grouping them into five perspectives: system objectives, user experience, infrastructure, model, and data. Together, these perspectives serve to mediate the communication between business owners, domain experts, designers, software and ML engineers, and data scientists. The creation of PerSpecML involved a series of validations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies. As a result of the diverse validations and continuous improvements, PerSpecML stands as a promising approach, poised to positively impact the specification of ML-enabled systems, particularly helping to reveal key components that would have been otherwise missed without using PerSpecML.