Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement

TL;DR

This paper introduces APL, an adaptive part learning method that enhances fine-grained category discovery by generating consistent object parts and a novel contrastive loss, improving discriminability and generalization without extra annotations.

Contribution

The paper presents a novel adaptive part discovery approach with a new contrastive loss, improving fine-grained category discovery by balancing discriminability and generalization.

Findings

Significant improvements on fine-grained datasets.

Effective integration with existing GCD frameworks.

Enhanced discriminability and generalization through part-based representations.

Abstract

Generalized Category Discovery (GCD) aims to recognize unlabeled images from known and novel classes by distinguishing novel classes from known ones, while also transferring knowledge from another set of labeled images with known classes. Existing GCD methods rely on self-supervised vision transformers such as DINO for representation learning. However, focusing solely on the global representation of the DINO CLS token introduces an inherent trade-off between discriminability and generalization. In this paper, we introduce an adaptive part discovery and learning method, called APL, which generates consistent object parts and their correspondences across different similar images using a set of shared learnable part queries and DINO part priors, without requiring any additional annotations. More importantly, we propose a novel all-min contrastive loss to learn discriminative yet generalizable part representation, which adaptively highlights discriminative object parts to distinguish similar categories for enhanced discriminability while simultaneously sharing other parts to facilitate knowledge transfer for improved generalization. Our APL can easily be incorporated into different GCD frameworks by replacing their CLS token feature with our part representations, showing significant enhancements on fine-grained datasets.