Learning Semantic Abstraction of Shape via 3D Region of Interest

TL;DR

This paper introduces a joint method for 3D shape abstraction and semantic analysis that produces instance-level semantic results, improving accuracy and applicability in shape segmentation and matching.

Contribution

A novel deep learning approach that simultaneously estimates 3D shape abstraction and semantic categories, addressing limitations of prior methods.

Findings

Achieves state-of-the-art results in 3D semantic analysis

Enables effective instance-level semantic part segmentation

Improves shape matching accuracy

Abstract

In this paper, we focus on the two tasks of 3D shape abstraction and semantic analysis. This is in contrast to current methods, which focus solely on either 3D shape abstraction or semantic analysis. In addition, previous methods have had difficulty producing instance-level semantic results, which has limited their application. We present a novel method for the joint estimation of a 3D shape abstraction and semantic analysis. Our approach first generates a number of 3D semantic candidate regions for a 3D shape; we then employ these candidates to directly predict the semantic categories and refine the parameters of the candidate regions simultaneously using a deep convolutional neural network. Finally, we design an algorithm to fuse the predicted results and obtain the final semantic abstraction, which is shown to be an improvement over a standard non maximum suppression. Experimental results demonstrate that our approach can produce state-of-the-art results. Moreover, we also find that our results can be easily applied to instance-level semantic part segmentation and shape matching.