A Deeper Look at 3D Shape Classifiers

TL;DR

This paper compares various 3D shape classification methods, analyzing their efficiency, generalization, and robustness, and introduces a new adversarial attack technique for multiview classifiers.

Contribution

It provides a comprehensive analysis of 3D shape classifiers, highlighting the effectiveness of multiview methods and proposing a novel adversarial perturbation approach.

Findings

Multiview methods outperform others in generalization without large-scale pretraining.

Cross-modal transfer improves voxel-based and point-based network performance.

Point-based networks exhibit greater robustness to point perturbations.

Abstract

We investigate the role of representations and architectures for classifying 3D shapes in terms of their computational efficiency, generalization, and robustness to adversarial transformations. By varying the number of training examples and employing cross-modal transfer learning we study the role of initialization of existing deep architectures for 3D shape classification. Our analysis shows that multiview methods continue to offer the best generalization even without pretraining on large labeled image datasets, and even when trained on simplified inputs such as binary silhouettes. Furthermore, the performance of voxel-based 3D convolutional networks and point-based architectures can be improved via cross-modal transfer from image representations. Finally, we analyze the robustness of 3D shape classifiers to adversarial transformations and present a novel approach for generating adversarial perturbations of a 3D shape for multiview classifiers using a differentiable renderer. We find that point-based networks are more robust to point position perturbations while voxel-based and multiview networks are easily fooled with the addition of imperceptible noise to the input.