Recurrent Segmentation for Variable Computational Budgets

TL;DR

This paper introduces a recurrent neural network for semantic image segmentation that adapts to different computational budgets and improves over iterations, enabling efficient video segmentation and competitive static image results.

Contribution

The paper proposes a recurrent segmentation model that operates across variable computational budgets and enhances prediction iteratively, unlike fixed-cost architectures.

Findings

Effective across a range of computational budgets.

Achieves near state-of-the-art performance on static datasets.

Reduces computational cost in video segmentation.

Abstract

State-of-the-art systems for semantic image segmentation use feed-forward pipelines with fixed computational costs. Building an image segmentation system that works across a range of computational budgets is challenging and time-intensive as new architectures must be designed and trained for every computational setting. To address this problem we develop a recurrent neural network that successively improves prediction quality with each iteration. Importantly, the RNN may be deployed across a range of computational budgets by merely running the model for a variable number of iterations. We find that this architecture is uniquely suited for efficiently segmenting videos. By exploiting the segmentation of past frames, the RNN can perform video segmentation at similar quality but reduced computational cost compared to state-of-the-art image segmentation methods. When applied to static images in the PASCAL VOC 2012 and Cityscapes segmentation datasets, the RNN traces out a speed-accuracy curve that saturates near the performance of state-of-the-art segmentation methods.