Mutation Testing framework for Machine Learning

TL;DR

This paper explores the evolution and current state of testing in Machine Learning Systems, emphasizing the need for reliable testing frameworks in critical applications to prevent severe consequences from model failures.

Contribution

It introduces a mutation testing framework tailored for Machine Learning, addressing the unique challenges of testing ML models in safety-critical domains.

Findings

Highlights the importance of reliable ML testing in critical sectors

Proposes a mutation testing approach for ML models

Discusses future directions for ML testing frameworks

Abstract

This is an article or technical note which is intended to provides an insight journey of Machine Learning Systems (MLS) testing, its evolution, current paradigm and future work. Machine Learning Models, used in critical applications such as healthcare industry, Automobile, and Air Traffic control, Share Trading etc., and failure of ML Model can lead to severe consequences in terms of loss of life or property. To remediate this, developers, scientists, and ML community around the world, must build a highly reliable test architecture for critical ML application. At the very foundation layer, any test model must satisfy the core testing attributes such as test properties and its components. This attribute comes from the software engineering, but the same cannot be applied in as-is form to the ML testing and we will tell you why.