Connectivity-constrained Interactive Panoptic Segmentation

TL;DR

This paper introduces connectivity-constrained algorithms for interactive panoptic segmentation, utilizing graph-based methods and ILP for optimal region segmentation, adaptable to various input features, and presents a scribble-based annotation framework.

Contribution

The paper proposes novel connectivity-enforcing segmentation algorithms and an interactive annotation framework that can leverage diverse feature inputs and ensure globally optimal segmentation.

Findings

ILP-based algorithm guarantees global optimality.

Algorithms work with RGB or deep feature maps.

Framework enables efficient interactive annotation.

Abstract

We address interactive panoptic annotation, where one segment all object and stuff regions in an image. We investigate two graph-based segmentation algorithms that both enforce connectivity of each region, with a notable class-aware Integer Linear Programming (ILP) formulation that ensures global optimum. Both algorithms can take RGB, or utilize the feature maps from any DCNN, whether trained on the target dataset or not, as input. We then propose an interactive, scribble-based annotation framework.