SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

TL;DR

This paper introduces SyncSpecCNN, a spectral convolutional neural network designed for 3D shape segmentation and keypoint prediction, effectively handling irregular shape graphs and achieving state-of-the-art results.

Contribution

The paper proposes a novel spectral CNN architecture with spectral parameterization and a spectral transformer to improve 3D shape analysis across different graphs.

Findings

Achieved state-of-the-art results on shape segmentation benchmarks.

Effectively handles irregular and diverse shape graphs.

Demonstrates robustness in keypoint prediction tasks.

Abstract

In this paper, we study the problem of semantic annotation on 3D models that are represented as shape graphs. A functional view is taken to represent localized information on graphs, so that annotations such as part segment or keypoint are nothing but 0-1 indicator vertex functions. Compared with images that are 2D grids, shape graphs are irregular and non-isomorphic data structures. To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases. Under this setting, our network, named SyncSpecCNN, strive to overcome two key challenges: how to share coefficients and conduct multi-scale analysis in different parts of the graph for a single shape, and how to share information across related but different shapes that may be represented by very different graphs. Towards these goals, we introduce a spectral parameterization of dilated convolutional kernels and a spectral transformer network. Experimentally we tested our SyncSpecCNN on various tasks, including 3D shape part segmentation and 3D keypoint prediction. State-of-the-art performance has been achieved on all benchmark datasets.