Towards Distribution-Agnostic Generalized Category Discovery

TL;DR

This paper introduces DA-GCD, a realistic task combining classifying known and unknown categories in imbalanced, open-world data, and proposes BaCon, a contrastive framework that effectively addresses this challenge.

Contribution

The paper defines the novel DA-GCD task and proposes BaCon, a contrastive learning framework with pseudo-labeling that jointly handles distribution imbalance and open-set recognition.

Findings

BaCon outperforms state-of-the-art methods on multiple datasets.

The contrastive-learning branch provides reliable distribution estimation.

Self-balanced knowledge transfer improves pseudo-label accuracy.

Abstract

Data imbalance and open-ended distribution are two intrinsic characteristics of the real visual world. Though encouraging progress has been made in tackling each challenge separately, few works dedicated to combining them towards real-world scenarios. While several previous works have focused on classifying close-set samples and detecting open-set samples during testing, it's still essential to be able to classify unknown subjects as human beings. In this paper, we formally define a more realistic task as distribution-agnostic generalized category discovery (DA-GCD): generating fine-grained predictions for both close- and open-set classes in a long-tailed open-world setting. To tackle the challenging problem, we propose a Self-Balanced Co-Advice contrastive framework (BaCon), which consists of a contrastive-learning branch and a pseudo-labeling branch, working collaboratively to provide interactive supervision to resolve the DA-GCD task. In particular, the contrastive-learning branch provides reliable distribution estimation to regularize the predictions of the pseudo-labeling branch, which in turn guides contrastive learning through self-balanced knowledge transfer and a proposed novel contrastive loss. We compare BaCon with state-of-the-art methods from two closely related fields: imbalanced semi-supervised learning and generalized category discovery. The effectiveness of BaCon is demonstrated with superior performance over all baselines and comprehensive analysis across various datasets. Our code is publicly available.