Annotation-Free Class-Incremental Learning

TL;DR

This paper introduces a new class-incremental learning paradigm that operates without labeled data, leveraging external world knowledge to enable models to learn new classes over time without supervision.

Contribution

The paper proposes AFCIL, a realistic continual learning setting without annotations, and introduces CrossWorld CL, a framework that uses external knowledge and cross-domain alignment to learn incrementally.

Findings

CrossWorld-CL outperforms CLIP baselines on four datasets.

The method effectively leverages external world knowledge.

Unsupervised semantic structure learning improves continual learning.

Abstract

Despite significant progress in continual learning ranging from architectural novelty to clever strategies for mitigating catastrophic forgetting most existing methods rest on a strong but unrealistic assumption the availability of labeled data throughout the learning process. In real-world scenarios, however, data often arrives sequentially and without annotations, rendering conventional approaches impractical. In this work, we revisit the fundamental assumptions of continual learning and ask: Can current systems adapt when labels are absent and tasks emerge incrementally over time? To this end, we introduce Annotation-Free Class-Incremental Learning (AFCIL), a more realistic and challenging paradigm where unlabeled data arrives continuously, and the learner must incrementally acquire new classes without any supervision. To enable effective learning under AFCIL, we propose CrossWorld CL, a Cross Domain World Guided Continual Learning framework that incorporates external world knowledge as a stable auxiliary source. The method retrieves semantically related ImageNet classes for each downstream category, maps downstream and ImageNet features through a cross domain alignment strategy and finally introduce a novel replay strategy. This design lets the model uncover semantic structure without annotations while keeping earlier knowledge intact. Across four datasets, CrossWorld-CL surpasses CLIP baselines and existing continual and unlabeled learning methods, underscoring the benefit of world knowledge for annotation free continual learning.