Recurrent Neural Networks for Semantic Instance Segmentation

TL;DR

This paper introduces a recurrent neural network model for semantic instance segmentation that generates object masks sequentially, eliminating the need for post-processing and demonstrating effectiveness across multiple benchmarks.

Contribution

The proposed end-to-end trainable recurrent model advances instance segmentation by removing reliance on object proposals and post-processing steps.

Findings

Effective on Pascal VOC 2012, CVPPP, and Cityscapes datasets.

Learns consistent object sorting patterns.

Correlates sorting patterns with encoder activations.

Abstract

We present a recurrent model for semantic instance segmentation that sequentially generates binary masks and their associated class probabilities for every object in an image. Our proposed system is trainable end-to-end from an input image to a sequence of labeled masks and, compared to methods relying on object proposals, does not require post-processing steps on its output. We study the suitability of our recurrent model on three different instance segmentation benchmarks, namely Pascal VOC 2012, CVPPP Plant Leaf Segmentation and Cityscapes. Further, we analyze the object sorting patterns generated by our model and observe that it learns to follow a consistent pattern, which correlates with the activations learned in the encoder part of our network. Source code and models are available at https://imatge-upc.github.io/rsis/