Open-Set Domain Adaptation Under Background Distribution Shift: Challenges and A Provably Efficient Solution

TL;DR

This paper introduces CoLOR, a novel method for open-set recognition that remains effective even when the background distribution shifts, supported by theoretical guarantees and extensive empirical validation.

Contribution

The paper develops CoLOR, a scalable and robust approach with provable guarantees for open-set recognition under background distribution shift, outperforming existing methods.

Findings

CoLOR outperforms baseline methods under background shift.

Theoretical guarantees are provided for CoLOR's effectiveness.

Performance is influenced by the size of the novel class.

Abstract

As we deploy machine learning systems in the real world, a core challenge is to maintain a model that is performant even as the data shifts. Such shifts can take many forms: new classes may emerge that were absent during training, a problem known as open-set recognition, and the distribution of known categories may change. Guarantees on open-set recognition are mostly derived under the assumption that the distribution of known classes, which we call the background distribution, is fixed. In this paper we develop CoLOR, a method that is guaranteed to solve open-set recognition even in the challenging case where the background distribution shifts. We prove that the method works under benign assumptions that the novel class is separable from the non-novel classes, and provide theoretical guarantees that it outperforms a representative baseline in a simplified overparameterized setting. We develop techniques to make CoLOR scalable and robust, and perform comprehensive empirical evaluations on image and text data. The results show that CoLOR significantly outperforms existing open-set recognition methods under background shift. Moreover, we provide new insights into how factors such as the size of the novel class influences performance, an aspect that has not been extensively explored in prior work.