Learning Canonical View Representation for 3D Shape Recognition with Arbitrary Views

TL;DR

This paper introduces a canonical view representation method for 3D shape recognition from arbitrary views, using optimal transport to align view features, leading to improved recognition performance in flexible viewing scenarios.

Contribution

We propose a novel canonical view representation approach that aligns arbitrary view features to learnable reference views, enabling robust 3D shape recognition from any viewpoint.

Findings

Achieves competitive results on ModelNet40 and ScanObjectNN datasets.

Significantly outperforms existing methods under arbitrary view settings.

Introduces a view separation constraint for better feature embedding.

Abstract

In this paper, we focus on recognizing 3D shapes from arbitrary views, i.e., arbitrary numbers and positions of viewpoints. It is a challenging and realistic setting for view-based 3D shape recognition. We propose a canonical view representation to tackle this challenge. We first transform the original features of arbitrary views to a fixed number of view features, dubbed canonical view representation, by aligning the arbitrary view features to a set of learnable reference view features using optimal transport. In this way, each 3D shape with arbitrary views is represented by a fixed number of canonical view features, which are further aggregated to generate a rich and robust 3D shape representation for shape recognition. We also propose a canonical view feature separation constraint to enforce that the view features in canonical view representation can be embedded into scattered points in a Euclidean space. Experiments on the ModelNet40, ScanObjectNN, and RGBD datasets show that our method achieves competitive results under the fixed viewpoint settings, and significantly outperforms the applicable methods under the arbitrary view setting.