AdaptIS: Adaptive Instance Selection Network

TL;DR

AdaptIS is a novel network architecture for class-agnostic instance segmentation that generates accurate masks for objects based on input points, adaptable to complex shapes and occlusions, and achieves state-of-the-art results on major benchmarks.

Contribution

It introduces an adaptive instance selection network using AdaIN layers for flexible, point-based object mask generation in segmentation tasks.

Findings

Achieves state-of-the-art results on Cityscapes and Mapillary datasets.

Performs competitively on the COCO dataset.

Effectively segments objects with complex shapes and occlusions.

Abstract

We present Adaptive Instance Selection network architecture for class-agnostic instance segmentation. Given an input image and a point $(x, y)$, it generates a mask for the object located at $(x, y)$. The network adapts to the input point with a help of AdaIN layers, thus producing different masks for different objects on the same image. AdaptIS generates pixel-accurate object masks, therefore it accurately segments objects of complex shape or severely occluded ones. AdaptIS can be easily combined with standard semantic segmentation pipeline to perform panoptic segmentation. To illustrate the idea, we perform experiments on a challenging toy problem with difficult occlusions. Then we extensively evaluate the method on panoptic segmentation benchmarks. We obtain state-of-the-art results on Cityscapes and Mapillary even without pretraining on COCO, and show competitive results on a challenging COCO dataset. The source code of the method and the trained models are available at https://github.com/saic-vul/adaptis.