Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions

TL;DR

This paper introduces a neural network approach to analyze and discover shared semantic structures in 3D shapes by learning consistent functional dictionaries from unaligned shape functions, enabling improved segmentation and keypoint detection.

Contribution

It presents a method to jointly analyze semantic probe functions on 3D shapes and learn consistent basis functions without correspondence information, capturing shared semantic structures.

Findings

Successfully learns functional dictionaries that reflect shared semantics.

Improves segmentation and keypoint selection on 3D models.

Works with shapes having independent discretizations and no known correspondences.

Abstract

Various 3D semantic attributes such as segmentation masks, geometric features, keypoints, and materials can be encoded as per-point probe functions on 3D geometries. Given a collection of related 3D shapes, we consider how to jointly analyze such probe functions over different shapes, and how to discover common latent structures using a neural network --- even in the absence of any correspondence information. Our network is trained on point cloud representations of shape geometry and associated semantic functions on that point cloud. These functions express a shared semantic understanding of the shapes but are not coordinated in any way. For example, in a segmentation task, the functions can be indicator functions of arbitrary sets of shape parts, with the particular combination involved not known to the network. Our network is able to produce a small dictionary of basis functions for each shape, a dictionary whose span includes the semantic functions provided for that shape. Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes. We demonstrate the effectiveness of our technique in various segmentation and keypoint selection applications.