Open Problems in Engineering and Quality Assurance of Safety Critical Machine Learning Systems

TL;DR

This paper identifies and organizes open problems in quality assurance for safety-critical machine learning systems, emphasizing the need for interdisciplinary approaches to develop effective standards, especially in automated-driving vehicles.

Contribution

It provides a structured classification of open problems in quality assurance for safety-critical ML systems, highlighting interdisciplinary challenges and industry trends.

Findings

Open problems are diverse and require interdisciplinary solutions.

Addressing these problems involves multiple fields like automotive, statistics, and software engineering.

Organizing open problems can facilitate industry-wide standard development.

Abstract

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems using machine-learning and deep-learning models, such as automated-driving vehicles. Quality assurance frameworks are required for such machine learning systems, but there are no widely accepted and established quality-assurance concepts and techniques. At the same time, open problems and the relevant technical fields are not organized. To establish standard quality assurance frameworks, it is necessary to visualize and organize these open problems in an interdisciplinary way, so that the experts from many different technical fields may discuss these problems in depth and develop solutions. In the present study, we identify, classify, and explore the open problems in quality assurance of safety-critical machine-learning systems, and their relevant corresponding industry and technological trends, using automated-driving vehicles as an example. Our results show that addressing these open problems requires incorporating knowledge from several different technological and industrial fields, including the automobile industry, statistics, software engineering, and machine learning.