Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentation

TL;DR

This paper introduces GradMix, a novel data augmentation technique that uses attribution maps to improve feature learning for open set recognition, enhancing detection of unseen classes and model robustness.

Contribution

GradMix is a new attribution-based augmentation method that dynamically masks learned concepts to promote diverse feature learning for open set recognition.

Findings

Outperforms state-of-the-art in open set recognition and OOD detection

Improves model robustness to corruptions

Enhances downstream classification in self-supervised learning

Abstract

Open set recognition (OSR) is devised to address the problem of detecting novel classes during model inference. Even in recent vision models, this remains an open issue which is receiving increasing attention. Thereby, a crucial challenge is to learn features that are relevant for unseen categories from given data, for which these features might not be discriminative. To facilitate this process and "optimize to learn" more diverse features, we propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training to mask out already learned concepts. Thus GradMix encourages the model to learn a more complete set of representative features from the same data source. Extensive experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method can often outperform the state-of-the-art. GradMix can further increase model robustness to corruptions as well as downstream classification performance for self-supervised learning, indicating its benefit for model generalization.