Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network

TL;DR

This paper introduces Relevance Flow, a novel method for interpreting hidden semantics in 3D point cloud neural networks, revealing learned features and enabling unsupervised segmentation and adversarial attacks.

Contribution

It presents Relevance Flow, a new approach to interpret intermediate layer semantics in 3D point cloud classifiers, uncovering plane and part-level features.

Findings

Reveals plane-level and part-level hidden semantics in intermediate layers.

Uses normal and IoU metrics to evaluate semantic consistency.

Enables unsupervised point cloud segmentation and adversarial attack generation.

Abstract

Although 3D point cloud classification neural network models have been widely used, the in-depth interpretation of the activation of the neurons and layers is still a challenge. We propose a novel approach, named Relevance Flow, to interpret the hidden semantics of 3D point cloud classification neural networks. It delivers the class Relevance to the activated neurons in the intermediate layers in a back-propagation manner, and associates the activation of neurons with the input points to visualize the hidden semantics of each layer. Specially, we reveal that the 3D point cloud classification neural network has learned the plane-level and part-level hidden semantics in the intermediate layers, and utilize the normal and IoU to evaluate the consistency of both levels' hidden semantics. Besides, by using the hidden semantics, we generate the adversarial attack samples to attack 3D point cloud classifiers. Experiments show that our proposed method reveals the hidden semantics of the 3D point cloud classification neural network on ModelNet40 and ShapeNet, which can be used for the unsupervised point cloud part segmentation without labels and attacking the 3D point cloud classifiers.