SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery

TL;DR

SEAL introduces a hierarchical, semantic-aware learning framework for generalized category discovery that leverages natural hierarchies and consistency modules to improve classification of known and unknown categories.

Contribution

The paper proposes a novel hierarchical semantic-guided contrastive learning method with a cross-granularity consistency module for improved GCD performance.

Findings

Achieves state-of-the-art results on multiple fine-grained benchmarks.

Effectively exploits hierarchical structures for better generalization.

Demonstrates robustness across coarse and fine-grained datasets.

Abstract

This paper investigates the problem of Generalized Category Discovery (GCD). Given a partially labelled dataset, GCD aims to categorize all unlabelled images, regardless of whether they belong to known or unknown classes. Existing approaches typically depend on either single-level semantics or manually designed abstract hierarchies, which limit their generalizability and scalability. To address these limitations, we introduce a SEmantic-aware hierArchical Learning framework (SEAL), guided by naturally occurring and easily accessible hierarchical structures. Within SEAL, we propose a Hierarchical Semantic-Guided Soft Contrastive Learning approach that exploits hierarchical similarity to generate informative soft negatives, addressing the limitations of conventional contrastive losses that treat all negatives equally. Furthermore, a Cross-Granularity Consistency (CGC) module is designed to align the predictions from different levels of granularity. SEAL consistently achieves state-of-the-art performance on fine-grained benchmarks, including the SSB benchmark, Oxford-Pet, and the Herbarium19 dataset, and further demonstrates generalization on coarse-grained datasets. Project page: https://visual-ai.github.io/seal/