Robust Novelty Detection through Style-Conscious Feature Ranking

TL;DR

This paper introduces Stylist, a method leveraging pretrained models to distinguish and discard style-biased features, thereby improving the robustness of novelty detection against style and content shifts.

Contribution

The paper proposes Stylist, a novel approach that uses feature distribution distances to filter out style-biased features, enhancing novelty detection robustness.

Findings

Stylist improves novelty detection performance across various datasets.

The method effectively discards environment-biased features.

Results show increased resilience to style and content shifts.

Abstract

Novelty detection seeks to identify samples deviating from a known distribution, yet data shifts in a multitude of ways, and only a few consist of relevant changes. Aligned with out-of-distribution generalization literature, we advocate for a formal distinction between task-relevant semantic or content changes and irrelevant style changes. This distinction forms the basis for robust novelty detection, emphasizing the identification of semantic changes resilient to style distributional shifts. To this end, we introduce Stylist, a method that utilizes pretrained large-scale model representations to selectively discard environment-biased features. By computing per-feature scores based on feature distribution distances between environments, Stylist effectively eliminates features responsible for spurious correlations, enhancing novelty detection performance. Evaluations on adapted domain generalization datasets and a synthetic dataset demonstrate Stylist's efficacy in improving novelty detection across diverse datasets with stylistic and content shifts. The code is available at https://github.com/bit-ml/Stylist.