Software Testing for Machine Learning

TL;DR

This paper reviews the current state and challenges of software testing in machine learning, emphasizing the need for reliable testing methods to ensure safety and correctness in critical applications.

Contribution

It identifies six key challenge areas in testing machine learning systems, analyzes existing approaches, and proposes a research agenda for future advancements.

Findings

Current testing approaches have significant limitations.

Six key challenge areas are critical for progress.

A research agenda is outlined for advancing testing methods.

Abstract

Machine learning has become prevalent across a wide variety of applications. Unfortunately, machine learning has also shown to be susceptible to deception, leading to errors, and even fatal failures. This circumstance calls into question the widespread use of machine learning, especially in safety-critical applications, unless we are able to assure its correctness and trustworthiness properties. Software verification and testing are established technique for assuring such properties, for example by detecting errors. However, software testing challenges for machine learning are vast and profuse - yet critical to address. This summary talk discusses the current state-of-the-art of software testing for machine learning. More specifically, it discusses six key challenge areas for software testing of machine learning systems, examines current approaches to these challenges and highlights their limitations. The paper provides a research agenda with elaborated directions for making progress toward advancing the state-of-the-art on testing of machine learning.