Localized Shortcut Removal

TL;DR

This paper introduces a novel adversarial method to detect and remove localized shortcuts in datasets, improving model generalization without sacrificing performance on clean data.

Contribution

It proposes a new approach using an adversarially trained lens to identify and neutralize localized shortcuts in images, enhancing dataset quality and model robustness.

Findings

Successfully detects and removes localized shortcuts in synthetic and real data

Maintains model performance on clean data after shortcut removal

Improves model generalization and robustness

Abstract

Machine learning is a data-driven field, and the quality of the underlying datasets plays a crucial role in learning success. However, high performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at hand. To address this issue for datasets where the shortcuts are smaller and more localized than true features, we propose a novel approach to detect and remove them. We use an adversarially trained lens to detect and eliminate highly predictive but semantically unconnected clues in images. In our experiments on both synthetic and real-world data, we show that our proposed approach reliably identifies and neutralizes such shortcuts without causing degradation of model performance on clean data. We believe that our approach can lead to more meaningful and generalizable machine learning models, especially in scenarios where the quality of the underlying datasets is crucial.