H-SPAM: Hierarchical Superpixel Anything Model

TL;DR

H-SPAM is a hierarchical superpixel model that generates accurate, nested, multi-scale image segmentations, outperforming existing methods in accuracy and regularity, with adjustable hierarchy via attention or user input.

Contribution

Introduces H-SPAM, a unified framework for hierarchical superpixels that improves accuracy, regularity, and multi-scale flexibility over prior approaches.

Findings

H-SPAM outperforms existing hierarchical superpixel methods in accuracy and regularity.

H-SPAM performs comparably to state-of-the-art non-hierarchical methods.

The hierarchy can be modulated using attention maps or user input.

Abstract

Superpixels offer a compact image representation by grouping pixels into coherent regions. Recent methods have reached a plateau in terms of segmentation accuracy by generating noisy superpixel shapes. Moreover, most existing approaches produce a single fixed-scale partition that limits their use in vision pipelines that would benefit multi-scale representations. In this work, we introduce H-SPAM (Hierarchical Superpixel Anything Model), a unified framework for generating accurate, regular, and perfectly nested hierarchical superpixels. Starting from a fine partition, guided by deep features and external object priors, H-SPAM constructs the hierarchy through a two-phase region merging process that first preserves object consistency and then allows controlled inter-object grouping. The hierarchy can also be modulated using visual attention maps or user input to preserve important regions longer in the hierarchy. Experiments on standard benchmarks show that H-SPAM strongly outperforms existing hierarchical methods in both accuracy and regularity, while performing on par with most recent state-of-the-art non-hierarchical methods. Code and pretrained models are available: https://github.com/waldo-j/hspam.