What is Software Quality for AI Engineers? Towards a Thinning of the Fog

TL;DR

This study explores software quality assurance strategies for AI/ML components in AI-enabled systems, identifying key issues, detection points, and providing insights to improve quality assurance practices.

Contribution

It offers a qualitative analysis of quality issues in AI/ML development and highlights when and how these issues are detected in real-world SME contexts.

Findings

Identified 12 key issues in AI/ML component development

Mapped when quality issues typically arise in the development process

Provided insights into detection methods for quality issues

Abstract

It is often overseen that AI-enabled systems are also software systems and therefore rely on software quality assurance (SQA). Thus, the goal of this study is to investigate the software quality assurance strategies adopted during the development, integration, and maintenance of AI/ML components and code. We conducted semi-structured interviews with representatives of ten Austrian SMEs that develop AI-enabled systems. A qualitative analysis of the interview data identified 12 issues in the development of AI/ML components. Furthermore, we identified when quality issues arise in AI/ML components and how they are detected. The results of this study should guide future work on software quality assurance processes and techniques for AI/ML components.