Towards Training-Free Open-World Classification with 3D Generative Models

TL;DR

This paper introduces a training-free, 3D generative model-based approach for open-world classification that is invariant to pose and category, significantly improving accuracy on benchmark datasets.

Contribution

It pioneers the use of 3D generative models for open-world classification, creating a rotation-invariant, training-free pipeline that outperforms existing methods.

Findings

Achieved 32.0% accuracy improvement on ModelNet10

Achieved 8.7% accuracy improvement on McGill

Demonstrated state-of-the-art performance in 3D open-world classification

Abstract

3D open-world classification is a challenging yet essential task in dynamic and unstructured real-world scenarios, requiring both open-category and open-pose recognition. To address these challenges, recent wisdom often takes sophisticated 2D pre-trained models to provide enriched and stable representations. However, these methods largely rely on how 3D objects can be projected into 2D space, which is unfortunately not well solved, and thus significantly limits their performance. Unlike these present efforts, in this paper we make a pioneering exploration of 3D generative models for 3D open-world classification. Drawing on abundant prior knowledge from 3D generative models, we additionally craft a rotation-invariant feature extractor. This innovative synergy endows our pipeline with the advantages of being training-free, open-category, and pose-invariant, thus well suited to 3D open-world classification. Extensive experiments on benchmark datasets demonstrate the potential of generative models in 3D open-world classification, achieving state-of-the-art performance on ModelNet10 and McGill with 32.0% and 8.7% overall accuracy improvement, respectively.