PiClick: Picking the desired mask from multiple candidates in click-based interactive segmentation

TL;DR

PiClick introduces a Transformer-based interactive segmentation method that generates multiple candidate masks and automatically suggests the most plausible one, reducing ambiguity and human effort in pixel-level annotation.

Contribution

The paper presents PiClick, a novel network that produces all potential masks and automatically selects the most suitable, addressing target ambiguity in click-based segmentation.

Findings

Outperforms previous methods on 9 datasets.

Effectively reduces human effort in mask selection.

Generates multiple masks for ambiguous scenes.

Abstract

Click-based interactive segmentation aims to generate target masks via human clicking, which facilitates efficient pixel-level annotation and image editing. In such a task, target ambiguity remains a problem hindering the accuracy and efficiency of segmentation. That is, in scenes with rich context, one click may correspond to multiple potential targets, while most previous interactive segmentors only generate a single mask and fail to deal with target ambiguity. In this paper, we propose a novel interactive segmentation network named PiClick, to yield all potentially reasonable masks and suggest the most plausible one for the user. Specifically, PiClick utilizes a Transformer-based architecture to generate all potential target masks by mutually interactive mask queries. Moreover, a Target Reasoning module(TRM) is designed in PiClick to automatically suggest the user-desired mask from all candidates, relieving target ambiguity and extra-human efforts. Extensive experiments on 9 interactive segmentation datasets demonstrate PiClick performs favorably against previous state-of-the-arts considering the segmentation results. Moreover, we show that PiClick effectively reduces human efforts in annotating and picking the desired masks. To ease the usage and inspire future research, we release the source code of PiClick together with a plug-and-play annotation tool at https://github.com/cilinyan/PiClick.