Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields

TL;DR

This paper introduces a joint CNN model that simultaneously predicts depth and intrinsic images from a single image, leveraging shared features and gradient domain inference to improve accuracy over previous methods.

Contribution

The novel joint convolutional neural field (JCNF) model effectively combines depth prediction and intrinsic image decomposition in a unified framework with shared features and gradient confidence learning.

Findings

Outperforms state-of-the-art on depth estimation

Achieves superior intrinsic image decomposition results

Demonstrates the effectiveness of joint learning in a single model

Abstract

We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.