Boundary IoU: Improving Object-Centric Image Segmentation Evaluation

TL;DR

This paper introduces Boundary IoU, a boundary-focused segmentation evaluation metric that better captures boundary errors, leading to more accurate assessment of segmentation quality, especially for large objects.

Contribution

The paper proposes Boundary IoU and associated metrics, updating evaluation protocols to emphasize boundary quality in segmentation tasks.

Findings

Boundary IoU is more sensitive to boundary errors than Mask IoU.

Boundary IoU does not over-penalize errors on small objects.

New metrics better track boundary quality improvements.

Abstract

We present Boundary IoU (Intersection-over-Union), a new segmentation evaluation measure focused on boundary quality. We perform an extensive analysis across different error types and object sizes and show that Boundary IoU is significantly more sensitive than the standard Mask IoU measure to boundary errors for large objects and does not over-penalize errors on smaller objects. The new quality measure displays several desirable characteristics like symmetry w.r.t. prediction/ground truth pairs and balanced responsiveness across scales, which makes it more suitable for segmentation evaluation than other boundary-focused measures like Trimap IoU and F-measure. Based on Boundary IoU, we update the standard evaluation protocols for instance and panoptic segmentation tasks by proposing the Boundary AP (Average Precision) and Boundary PQ (Panoptic Quality) metrics, respectively. Our experiments show that the new evaluation metrics track boundary quality improvements that are generally overlooked by current Mask IoU-based evaluation metrics. We hope that the adoption of the new boundary-sensitive evaluation metrics will lead to rapid progress in segmentation methods that improve boundary quality.