GraCo: Granularity-Controllable Interactive Segmentation

TL;DR

GraCo introduces a flexible interactive segmentation method that allows precise control over output granularity, reducing redundancy and ambiguity, by leveraging an automatic mask generator and a novel learning strategy.

Contribution

The paper presents GraCo, a new approach enabling controllable granularity in interactive segmentation through automatic mask generation and a specialized training strategy.

Findings

Outperforms previous methods in complex object and part segmentation scenarios.

Effectively controls segmentation granularity with minimal annotation effort.

Demonstrates potential as a versatile annotation tool for diverse tasks.

Abstract

Interactive Segmentation (IS) segments specific objects or parts in the image according to user input. Current IS pipelines fall into two categories: single-granularity output and multi-granularity output. The latter aims to alleviate the spatial ambiguity present in the former. However, the multi-granularity output pipeline suffers from limited interaction flexibility and produces redundant results. In this work, we introduce Granularity-Controllable Interactive Segmentation (GraCo), a novel approach that allows precise control of prediction granularity by introducing additional parameters to input. This enhances the customization of the interactive system and eliminates redundancy while resolving ambiguity. Nevertheless, the exorbitant cost of annotating multi-granularity masks and the lack of available datasets with granularity annotations make it difficult for models to acquire the necessary guidance to control output granularity. To address this problem, we design an any-granularity mask generator that exploits the semantic property of the pre-trained IS model to automatically generate abundant mask-granularity pairs without requiring additional manual annotation. Based on these pairs, we propose a granularity-controllable learning strategy that efficiently imparts the granularity controllability to the IS model. Extensive experiments on intricate scenarios at object and part levels demonstrate that our GraCo has significant advantages over previous methods. This highlights the potential of GraCo to be a flexible annotation tool, capable of adapting to diverse segmentation scenarios. The project page: https://zhao-yian.github.io/GraCo.