Recurrent Convolutional Neural Networks for Scene Parsing

TL;DR

This paper introduces a recurrent convolutional neural network for scene parsing that captures long-range dependencies efficiently, achieves state-of-the-art results without segmentation or task-specific features, and maintains fast inference.

Contribution

The proposed recurrent CNN effectively models large context in scene parsing without relying on segmentation or task-specific features, enabling end-to-end training and improved accuracy.

Findings

State-of-the-art performance on Stanford Background Dataset

State-of-the-art performance on SIFT Flow Dataset

Fast inference at test time

Abstract

Scene parsing is a technique that consist on giving a label to all pixels in an image according to the class they belong to. To ensure a good visual coherence and a high class accuracy, it is essential for a scene parser to capture image long range dependencies. In a feed-forward architecture, this can be simply achieved by considering a sufficiently large input context patch, around each pixel to be labeled. We propose an approach consisting of a recurrent convolutional neural network which allows us to consider a large input context, while limiting the capacity of the model. Contrary to most standard approaches, our method does not rely on any segmentation methods, nor any task-specific features. The system is trained in an end-to-end manner over raw pixels, and models complex spatial dependencies with low inference cost. As the context size increases with the built-in recurrence, the system identifies and corrects its own errors. Our approach yields state-of-the-art performance on both the Stanford Background Dataset and the SIFT Flow Dataset, while remaining very fast at test time.