Fine-grained Metrics for Point Cloud Semantic Segmentation

TL;DR

This paper introduces fine-grained evaluation metrics for point cloud semantic segmentation to better address category and size imbalances, providing more detailed insights into model performance across diverse object types and sizes.

Contribution

It proposes new fine-grained mIoU and mAcc metrics that reduce bias towards large objects and offer richer statistical analysis for segmentation models.

Findings

Fine-grained metrics reveal detailed model performance differences.

Metrics reduce bias towards large objects in datasets.

Evaluation on multiple datasets demonstrates improved assessment.

Abstract

Two forms of imbalances are commonly observed in point cloud semantic segmentation datasets: (1) category imbalances, where certain objects are more prevalent than others; and (2) size imbalances, where certain objects occupy more points than others. Because of this, the majority of categories and large objects are favored in the existing evaluation metrics. This paper suggests fine-grained mIoU and mAcc for a more thorough assessment of point cloud segmentation algorithms in order to address these issues. Richer statistical information is provided for models and datasets by these fine-grained metrics, which also lessen the bias of current semantic segmentation metrics towards large objects. The proposed metrics are used to train and assess various semantic segmentation algorithms on three distinct indoor and outdoor semantic segmentation datasets.