Representing Shape Collections with Alignment-Aware Linear Models

TL;DR

This paper introduces a deep learning approach to represent 3D shape collections using alignment-aware linear models, enabling explicit, interpretable, and visualizable shape representations that achieve state-of-the-art few-shot segmentation results.

Contribution

The paper presents a novel method combining deep learning with linear shape models for 3D point clouds, emphasizing explicit, interpretable representations learned end-to-end.

Findings

Achieves state-of-the-art few-shot segmentation performance.

Provides explicit and visualizable shape models.

Enables understanding of failure cases through visualization.

Abstract

In this paper, we revisit the classical representation of 3D point clouds as linear shape models. Our key insight is to leverage deep learning to represent a collection of shapes as affine transformations of low-dimensional linear shape models. Each linear model is characterized by a shape prototype, a low-dimensional shape basis and two neural networks. The networks take as input a point cloud and predict the coordinates of a shape in the linear basis and the affine transformation which best approximate the input. Both linear models and neural networks are learned end-to-end using a single reconstruction loss. The main advantage of our approach is that, in contrast to many recent deep approaches which learn feature-based complex shape representations, our model is explicit and every operation occurs in 3D space. As a result, our linear shape models can be easily visualized and annotated, and failure cases can be visually understood. While our main goal is to introduce a compact and interpretable representation of shape collections, we show it leads to state of the art results for few-shot segmentation.