Generalized Class Discovery in Instance Segmentation

TL;DR

This paper introduces a novel approach for generalized class discovery in instance segmentation, effectively handling imbalanced data and leveraging pseudo-labels to identify both known and novel object categories.

Contribution

The work proposes an instance-wise temperature assignment method, reliability criteria for pseudo-labels, and an attention module, advancing the state-of-the-art in generalized class discovery for instance segmentation.

Findings

Outperforms previous methods on COCO$_{half}$ + LVIS and LVIS + Visual Genome datasets.

Effectively handles long-tailed class distributions in instance segmentation.

Improves discovery of both known and novel classes with pseudo-label reliability strategies.

Abstract

This work addresses the task of generalized class discovery (GCD) in instance segmentation. The goal is to discover novel classes and obtain a model capable of segmenting instances of both known and novel categories, given labeled and unlabeled data. Since the real world contains numerous objects with long-tailed distributions, the instance distribution for each class is inherently imbalanced. To address the imbalanced distributions, we propose an instance-wise temperature assignment (ITA) method for contrastive learning and class-wise reliability criteria for pseudo-labels. The ITA method relaxes instance discrimination for samples belonging to head classes to enhance GCD. The reliability criteria are to avoid excluding most pseudo-labels for tail classes when training an instance segmentation network using pseudo-labels from GCD. Additionally, we propose dynamically adjusting the criteria to leverage diverse samples in the early stages while relying only on reliable pseudo-labels in the later stages. We also introduce an efficient soft attention module to encode object-specific representations for GCD. Finally, we evaluate our proposed method by conducting experiments on two settings: COCO$_{half}$ + LVIS and LVIS + Visual Genome. The experimental results demonstrate that the proposed method outperforms previous state-of-the-art methods.