A Novel Patch Convolutional Neural Network for View-based 3D Model Retrieval

TL;DR

This paper introduces a novel patch convolutional neural network (PCNN) that captures long-range relationships among multi-view images for improved view-based 3D model retrieval, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a new PCNN architecture that exploits patch relationships and adaptive view weighting to enhance 3D model retrieval accuracy.

Findings

Achieves state-of-the-art mAP of 93.67% on ModelNet40

Outperforms existing methods on ModelNet10

Effectively captures long-range view associations

Abstract

Recently, many view-based 3D model retrieval methods have been proposed and have achieved state-of-the-art performance. Most of these methods focus on extracting more discriminative view-level features and effectively aggregating the multi-view images of a 3D model, but the latent relationship among these multi-view images is not fully explored. Thus, we tackle this problem from the perspective of exploiting the relationships between patch features to capture long-range associations among multi-view images. To capture associations among views, in this work, we propose a novel patch convolutional neural network (PCNN) for view-based 3D model retrieval. Specifically, we first employ a CNN to extract patch features of each view image separately. Secondly, a novel neural network module named PatchConv is designed to exploit intrinsic relationships between neighboring patches in the feature space to capture long-range associations among multi-view images. Then, an adaptive weighted view layer is further embedded into PCNN to automatically assign a weight to each view according to the similarity between each view feature and the view-pooling feature. Finally, a discrimination loss function is employed to extract the discriminative 3D model feature, which consists of softmax loss values generated by the fusion lassifier and the specific classifier. Extensive experimental results on two public 3D model retrieval benchmarks, namely, the ModelNet40, and ModelNet10, demonstrate that our proposed PCNN can outperform state-of-the-art approaches, with mAP alues of 93.67%, and 96.23%, respectively.