Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy

TL;DR

Defuse is a novel method that automatically discovers, categorizes, and corrects model errors beyond test data by generating misclassifications using generative models, thereby improving model robustness and trustworthiness.

Contribution

The paper introduces Defuse, a technique that leverages generative models to find and fix unseen model errors, enhancing debugging beyond traditional test accuracy measures.

Findings

Defuse identifies specific error regions in generative model latent space.

It categorizes errors into high-level bugs for targeted correction.

Applying Defuse improves classifier accuracy on discovered errors while preserving test performance.

Abstract

We typically compute aggregate statistics on held-out test data to assess the generalization of machine learning models. However, statistics on test data often overstate model generalization, and thus, the performance of deployed machine learning models can be variable and untrustworthy. Motivated by these concerns, we develop methods to automatically discover and correct model errors beyond those available in the data. We propose Defuse, a method that generates novel model misclassifications, categorizes these errors into high-level model bugs, and efficiently labels and fine-tunes on the errors to correct them. To generate misclassified data, we propose an algorithm inspired by adversarial machine learning techniques that uses a generative model to find naturally occurring instances misclassified by a model. Further, we observe that the generative models have regions in their latent space with higher concentrations of misclassifications. We call these regions misclassification regions and find they have several useful properties. Each region contains a specific type of model bug; for instance, a misclassification region for an MNIST classifier contains a style of skinny 6 that the model mistakes as a 1. We can also assign a single label to each region, facilitating low-cost labeling. We propose a method to learn the misclassification regions and use this insight to both categorize errors and correct them. In practice, Defuse finds and corrects novel errors in classifiers. For example, Defuse shows that a high-performance traffic sign classifier mistakes certain 50km/h signs as 80km/h. Defuse corrects the error after fine-tuning while maintaining generalization on the test set.