From Machine Learning Documentation to Requirements: Bridging Processes with Requirements Languages

TL;DR

This paper explores how ML documentation like ModelCards and DataSheets can be mined for requirements information and structured into formal requirements representations to improve ML system development processes.

Contribution

It investigates the RE-relevant content in ML documentation and evaluates the effectiveness of existing RE templates to structure this knowledge.

Findings

ML documentation contains significant RE-relevant information.

Established RE templates can effectively structure ML documentation data.

A pathway is demonstrated for integrating ML documentation into requirements engineering.

Abstract

In software engineering processes for machine learning (ML)-enabled systems, integrating and verifying ML components is a major challenge. A prerequisite is the specification of ML component requirements, including models and data, an area where traditional requirements engineering (RE) processes face new obstacles. An underexplored source of RE-relevant information in this context is ML documentation such as ModelCards and DataSheets. However, it is uncertain to what extent RE-relevant information can be extracted from these documents. This study first investigates the amount and nature of RE-relevant information in 20 publicly available ModelCards and DataSheets. We show that these documents contain a significant amount of potentially RE-relevant information. Next, we evaluate how effectively three established RE representations (EARS, Rupp's template, and Volere) can structure this knowledge into requirements. Our results demonstrate that there is a pathway to transform ML-specific knowledge into structured requirements, incorporating ML documentation in software engineering processes for ML systems.