Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs

TL;DR

This paper introduces a novel densely connected Markov random field model for pixel-wise instance segmentation in autonomous driving, improving global consistency and accuracy over previous patch-based methods.

Contribution

It formulates a global labeling problem with a new densely connected MRF and demonstrates efficient mean field inference for improved instance segmentation.

Findings

Significant performance boost on KITTI benchmark

Effective encoding of patch predictions and region separation

Enhanced global consistency in instance labeling

Abstract

Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving. We build on recent work [Zhang et al., ICCV15] that trained a convolutional neural net to predict instance labeling in local image patches, extracted exhaustively in a stride from an image. A simple Markov random field model using several heuristics was then proposed in [Zhang et al., ICCV15] to derive a globally consistent instance labeling of the image. In this paper, we formulate the global labeling problem with a novel densely connected Markov random field and show how to encode various intuitive potentials in a way that is amenable to efficient mean field inference [Kr\"ahenb\"uhl et al., NIPS11]. Our potentials encode the compatibility between the global labeling and the patch-level predictions, contrast-sensitive smoothness as well as the fact that separate regions form different instances. Our experiments on the challenging KITTI benchmark [Geiger et al., CVPR12] demonstrate that our method achieves a significant performance boost over the baseline [Zhang et al., ICCV15].