Towards Explainable LiDAR Point Cloud Semantic Segmentation via Gradient Based Target Localization

TL;DR

This paper presents pGS-CAM, a gradient-based method for generating saliency maps to interpret LiDAR point cloud semantic segmentation models, improving understanding of model decisions across multiple datasets and architectures.

Contribution

Introduction of pGS-CAM, a robust gradient-based saliency method for interpreting 3D point cloud segmentation models, inspired by Grad-CAM, applicable to various datasets and architectures.

Findings

pGS-CAM effectively highlights important points in LiDAR data.

The method is robust across datasets like SemanticKITTI and Paris-Lille3D.

It improves interpretability of 3D segmentation models.

Abstract

Semantic Segmentation (SS) of LiDAR point clouds is essential for many applications, such as urban planning and autonomous driving. While much progress has been made in interpreting SS predictions for images, interpreting point cloud SS predictions remains a challenge. This paper introduces pGS-CAM, a novel gradient-based method for generating saliency maps in neural network activation layers. Inspired by Grad-CAM, which uses gradients to highlight local importance, pGS-CAM is robust and effective on a variety of datasets (SemanticKITTI, Paris-Lille3D, DALES) and 3D deep learning architectures (KPConv, RandLANet). Our experiments show that pGS-CAM effectively accentuates the feature learning in intermediate activations of SS architectures by highlighting the contribution of each point. This allows us to better understand how SS models make their predictions and identify potential areas for improvement. Relevant codes are available at https://github.com/geoai4cities/pGS-CAM.