Segmentation Similarity and Agreement

TL;DR

This paper introduces a new segmentation similarity metric, S, that measures how closely two segmentations match by considering boundary transformations, and adapts agreement coefficients for segmentation evaluation, improving upon existing methods.

Contribution

The paper presents a novel segmentation similarity metric and adapted agreement coefficients, enhancing segmentation evaluation accuracy and flexibility compared to prior approaches.

Findings

S effectively measures segmentation similarity using edit distance.

Adapted agreement coefficients are suitable for various segmentation tasks.

S outperforms existing segmentation evaluation metrics.

Abstract

We propose a new segmentation evaluation metric, called segmentation similarity (S), that quantifies the similarity between two segmentations as the proportion of boundaries that are not transformed when comparing them using edit distance, essentially using edit distance as a penalty function and scaling penalties by segmentation size. We propose several adapted inter-annotator agreement coefficients which use S that are suitable for segmentation. We show that S is configurable enough to suit a wide variety of segmentation evaluations, and is an improvement upon the state of the art. We also propose using inter-annotator agreement coefficients to evaluate automatic segmenters in terms of human performance.