PartCom: Part Composition Learning for 3D Open-Set Recognition

TL;DR

This paper introduces PartCom, a novel 3D open-set recognition method using part prototypes and feature synthesis to effectively identify known and unknown classes, improving safety in applications like autonomous driving.

Contribution

The paper proposes a part prototype-based approach with a PUFS module for synthesizing unknown features, addressing open-set risks in 3D recognition tasks.

Findings

PartCom outperforms state-of-the-art methods on 3D OSR benchmarks.

Part prototypes effectively represent 3D shapes for open-set recognition.

Synthesizing unknown features improves detection of unknown classes.

Abstract

3D recognition is the foundation of 3D deep learning in many emerging fields, such as autonomous driving and robotics.Existing 3D methods mainly focus on the recognition of a fixed set of known classes and neglect possible unknown classes during testing. These unknown classes may cause serious accidents in safety-critical applications, i.e. autonomous driving. In this work, we make a first attempt to address 3D open-set recognition (OSR) so that a classifier can recognize known classes as well as be aware of unknown classes. We analyze open-set risks in the 3D domain and point out the overconfidence and under-representation problems that make existing methods perform poorly on the 3D OSR task. To resolve above problems, we propose a novel part prototype-based OSR method named PartCom. We use part prototypes to represent a 3D shape as a part composition, since a part composition can represent the overall structure of a shape and can help distinguish different known classes and unknown ones. Then we formulate two constraints on part prototypes to ensure their effectiveness. To reduce open-set risks further, we devise a PUFS module to synthesize unknown features as representatives of unknown samples by mixing up part composite features of different classes. We conduct experiments on three kinds of 3D OSR tasks based on both CAD shape dataset and scan shape dataset. Extensive experiments show that our method is powerful in classifying known classes and unknown ones and can attain much better results than SOTA baselines on all 3D OSR tasks. The project will be released.