Machine Learning Testing: Survey, Landscapes and Horizons

TL;DR

This survey comprehensively reviews Machine Learning Testing research, covering properties, components, workflows, and applications, analyzing trends and identifying future challenges and directions in the field.

Contribution

It provides an extensive overview of ML testing research, synthesizing 144 papers and highlighting key properties, workflows, and future research challenges.

Findings

Identified key testing properties like correctness, robustness, and fairness.

Analyzed trends in datasets and research focus areas.

Outlined future research challenges and promising directions.

Abstract

This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 144 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing.