SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images

TL;DR

This paper introduces SPIN, a hierarchical semantic segmentation dataset with subpart annotations for natural images, along with new evaluation metrics and benchmarking of modern models across multiple tasks.

Contribution

It presents the first hierarchical segmentation dataset with subpart annotations and novel metrics, enabling better evaluation of models' understanding of hierarchical structures.

Findings

Models vary in capturing hierarchical relationships.

Benchmark results highlight strengths and weaknesses of current methods.

The dataset facilitates future research in hierarchical segmentation.

Abstract

Hierarchical segmentation entails creating segmentations at varying levels of granularity. We introduce the first hierarchical semantic segmentation dataset with subpart annotations for natural images, which we call SPIN (SubPartImageNet). We also introduce two novel evaluation metrics to evaluate how well algorithms capture spatial and semantic relationships across hierarchical levels. We benchmark modern models across three different tasks and analyze their strengths and weaknesses across objects, parts, and subparts. To facilitate community-wide progress, we publicly release our dataset at https://joshmyersdean.github.io/spin/index.html.