Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions

TL;DR

This paper introduces a new fine-grained 3D shape dataset and proposes FG3D-Net, a method that captures local details using hierarchical part-view attention and RNNs, significantly improving classification accuracy.

Contribution

The paper presents a novel fine-grained 3D shape dataset and a hierarchical part-view attention model that enhances discriminability by focusing on important parts and views.

Findings

FG3D-Net outperforms state-of-the-art methods on the new dataset.

Hierarchical attention improves local detail capture.

Semantic part detection aids in fine-grained classification.

Abstract

Fine-grained 3D shape classification is important for shape understanding and analysis, which poses a challenging research problem. However, the studies on the fine-grained 3D shape classification have rarely been explored, due to the lack of fine-grained 3D shape benchmarks. To address this issue, we first introduce a new 3D shape dataset (named FG3D dataset) with fine-grained class labels, which consists of three categories including airplane, car and chair. Each category consists of several subcategories at a fine-grained level. According to our experiments under this fine-grained dataset, we find that state-of-the-art methods are significantly limited by the small variance among subcategories in the same category. To resolve this problem, we further propose a novel fine-grained 3D shape classification method named FG3D-Net to capture the fine-grained local details of 3D shapes from multiple rendered views. Specifically, we first train a Region Proposal Network (RPN) to detect the generally semantic parts inside multiple views under the benchmark of generally semantic part detection. Then, we design a hierarchical part-view attention aggregation module to learn a global shape representation by aggregating generally semantic part features, which preserves the local details of 3D shapes. The part-view attention module hierarchically leverages part-level and view-level attention to increase the discriminability of our features. The part-level attention highlights the important parts in each view while the view-level attention highlights the discriminative views among all the views of the same object. In addition, we integrate a Recurrent Neural Network (RNN) to capture the spatial relationships among sequential views from different viewpoints. Our results under the fine-grained 3D shape dataset show that our method outperforms other state-of-the-art methods.