OpenGCD: Assisting Open World Recognition with Generalized Category Discovery

TL;DR

OpenGCD introduces an integrated system for open world recognition that automates unknown class grouping and incremental learning, significantly improving performance on standard benchmarks by combining uncertainty scoring, generalized category discovery, and exemplar retention.

Contribution

It is the first to incorporate generalized category discovery into open world recognition, automating unknown class grouping and enhancing incremental learning.

Findings

Outperforms baseline methods on benchmark datasets.

Introduces a new harmonic clustering accuracy metric.

Demonstrates effective automation of open world recognition tasks.

Abstract

A desirable open world recognition (OWR) system requires performing three tasks: (1) Open set recognition (OSR), i.e., classifying the known (classes seen during training) and rejecting the unknown (unseen$/$novel classes) online; (2) Grouping and labeling these unknown as novel known classes; (3) Incremental learning (IL), i.e., continual learning these novel classes and retaining the memory of old classes. Ideally, all of these steps should be automated. However, existing methods mostly assume that the second task is completely done manually. To bridge this gap, we propose OpenGCD that combines three key ideas to solve the above problems sequentially: (a) We score the origin of instances (unknown or specifically known) based on the uncertainty of the classifier's prediction; (b) For the first time, we introduce generalized category discovery (GCD) techniques in OWR to assist humans in grouping unlabeled data; (c) For the smooth execution of IL and GCD, we retain an equal number of informative exemplars for each class with diversity as the goal. Moreover, we present a new performance evaluation metric for GCD called harmonic clustering accuracy. Experiments on two standard classification benchmarks and a challenging dataset demonstrate that OpenGCD not only offers excellent compatibility but also substantially outperforms other baselines. Code: https://github.com/Fulin-Gao/OpenGCD.