Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture

TL;DR

This paper presents a versatile multi-scale convolutional neural network that simultaneously predicts depth, surface normals, and semantic labels from images, achieving state-of-the-art results across all tasks.

Contribution

The authors introduce a unified multi-scale CNN architecture adaptable to multiple vision tasks with minimal modifications, eliminating the need for superpixels or segmentation.

Findings

Achieved state-of-the-art performance on depth prediction benchmarks.

Successfully estimated surface normals with high accuracy.

Performed semantic labeling effectively across diverse datasets.

Abstract

In this paper we address three different computer vision tasks using a single basic architecture: depth prediction, surface normal estimation, and semantic labeling. We use a multiscale convolutional network that is able to adapt easily to each task using only small modifications, regressing from the input image to the output map directly. Our method progressively refines predictions using a sequence of scales, and captures many image details without any superpixels or low-level segmentation. We achieve state-of-the-art performance on benchmarks for all three tasks.