Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space

TL;DR

This paper proposes a method to estimate the robustness of classification models by analyzing the structure of their learned feature-space, using unsupervised clustering to predict performance on corrupted data.

Contribution

It introduces robustness indicators based on feature-space clustering, providing a new way to assess model robustness beyond fixed test sets.

Findings

High correlation between clustering-based robustness indicators and model performance on corrupted data.

Method offers a more comprehensive robustness assessment than traditional fixed test benchmarks.

Applicable to various trained classifiers for robustness estimation.

Abstract

Over the last decade, the development of deep image classification networks has mostly been driven by the search for the best performance in terms of classification accuracy on standardized benchmarks like ImageNet. More recently, this focus has been expanded by the notion of model robustness, \ie the generalization abilities of models towards previously unseen changes in the data distribution. While new benchmarks, like ImageNet-C, have been introduced to measure robustness properties, we argue that fixed testsets are only able to capture a small portion of possible data variations and are thus limited and prone to generate new overfitted solutions. To overcome these drawbacks, we suggest to estimate the robustness of a model directly from the structure of its learned feature-space. We introduce robustness indicators which are obtained via unsupervised clustering of latent representations from a trained classifier and show very high correlations to the model performance on corrupted test data.