Monocular Object Instance Segmentation and Depth Ordering with CNNs

TL;DR

This paper presents a CNN-based method for simultaneous instance segmentation and depth ordering from a single image, using a Markov random field to unify patch-based predictions, achieving strong results on the KITTI benchmark.

Contribution

The paper introduces a novel CNN and MRF framework for joint instance segmentation and depth ordering from monocular images, improving accuracy on challenging datasets.

Findings

Effective joint segmentation and depth ordering achieved

Strong performance demonstrated on KITTI benchmark

Patch-based CNN predictions refined by MRF improve results

Abstract

In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image. Towards this goal, we take advantage of convolutional neural nets and train them to directly predict instance-level segmentations where the instance ID encodes the depth ordering within image patches. To provide a coherent single explanation of an image we develop a Markov random field which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm. It aims to predict accurate instance-level segmentation and depth ordering. We demonstrate the effectiveness of our approach on the challenging KITTI benchmark and show good performance on both tasks.