Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling

TL;DR

This paper introduces a fully convolutional network-based method that combines semantic labeling, depth prediction, and instance encoding to achieve state-of-the-art instance segmentation and distance estimation in street scenes.

Contribution

The proposed approach uniquely integrates pixel-level encoding and depth layering within a fully convolutional network for improved instance segmentation.

Findings

Outperforms existing methods on KITTI and Cityscapes datasets

Achieves accurate instance segmentation and distance estimation

Provides a unified framework for semantic, depth, and instance labeling

Abstract

Recent approaches for instance-aware semantic labeling have augmented convolutional neural networks (CNNs) with complex multi-task architectures or computationally expensive graphical models. We present a method that leverages a fully convolutional network (FCN) to predict semantic labels, depth and an instance-based encoding using each pixel's direction towards its corresponding instance center. Subsequently, we apply low-level computer vision techniques to generate state-of-the-art instance segmentation on the street scene datasets KITTI and Cityscapes. Our approach outperforms existing works by a large margin and can additionally predict absolute distances of individual instances from a monocular image as well as a pixel-level semantic labeling.