NeurNCD: Novel Class Discovery via Implicit Neural Representation

TL;DR

NeurNCD introduces a novel, data-efficient implicit neural representation framework for discovering new classes in open-world scenarios, outperforming existing methods without requiring dense labels.

Contribution

The paper presents NeurNCD, a versatile framework combining Embedding-NeRF and KL divergence to improve novel class discovery using implicit neural representations.

Findings

Achieves superior segmentation performance on NYUv2 and Replica datasets.

Operates effectively without densely labeled datasets or human supervision.

Outperforms state-of-the-art approaches in open and closed-world settings.

Abstract

Discovering novel classes in open-world settings is crucial for real-world applications. Traditional explicit representations, such as object descriptors or 3D segmentation maps, are constrained by their discrete, hole-prone, and noisy nature, which hinders accurate novel class discovery. To address these challenges, we introduce NeurNCD, the first versatile and data-efficient framework for novel class discovery that employs the meticulously designed Embedding-NeRF model combined with KL divergence as a substitute for traditional explicit 3D segmentation maps to aggregate semantic embedding and entropy in visual embedding space. NeurNCD also integrates several key components, including feature query, feature modulation and clustering, facilitating efficient feature augmentation and information exchange between the pre-trained semantic segmentation network and implicit neural representations. As a result, our framework achieves superior segmentation performance in both open and closed-world settings without relying on densely labelled datasets for supervised training or human interaction to generate sparse label supervision. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches on the NYUv2 and Replica datasets.