Proto-FG3D: Prototype-based Interpretable Fine-Grained 3D Shape Classification

TL;DR

Proto-FG3D introduces a prototype-based framework for fine-grained 3D shape classification, improving accuracy, interpretability, and robustness by shifting from parametric to non-parametric learning and utilizing multi-view, multi-category representations.

Contribution

It is the first to propose a prototype-based approach for fine-grained 3D shape classification, enhancing interpretability and discriminative power over existing parametric methods.

Findings

Outperforms state-of-the-art in accuracy on FG3D and ModelNet40

Provides transparent, case-based interpretability

Improves robustness and class balance through online clustering

Abstract

Deep learning-based multi-view coarse-grained 3D shape classification has achieved remarkable success over the past decade, leveraging the powerful feature learning capabilities of CNN-based and ViT-based backbones. However, as a challenging research area critical for detailed shape understanding, fine-grained 3D classification remains understudied due to the limited discriminative information captured during multi-view feature aggregation, particularly for subtle inter-class variations, class imbalance, and inherent interpretability limitations of parametric model. To address these problems, we propose the first prototype-based framework named Proto-FG3D for fine-grained 3D shape classification, achieving a paradigm shift from parametric softmax to non-parametric prototype learning. Firstly, Proto-FG3D establishes joint multi-view and multi-category representation learning via Prototype Association. Secondly, prototypes are refined via Online Clustering, improving both the robustness of multi-view feature allocation and inter-subclass balance. Finally, prototype-guided supervised learning is established to enhance fine-grained discrimination via prototype-view correlation analysis and enables ad-hoc interpretability through transparent case-based reasoning. Experiments on FG3D and ModelNet40 show Proto-FG3D surpasses state-of-the-art methods in accuracy, transparent predictions, and ad-hoc interpretability with visualizations, challenging conventional fine-grained 3D recognition approaches.