DARN: a Deep Adversial Residual Network for Intrinsic Image Decomposition

TL;DR

This paper introduces DARN, a deep adversarial residual network that performs intrinsic image decomposition into albedo and shading without relying on physical priors, using a fully convolutional, end-to-end trainable architecture with adversarial training.

Contribution

It presents a novel fully convolutional neural network architecture for intrinsic decomposition that is data-driven, simpler, and more effective than previous methods, eliminating the need for physical priors.

Findings

Outperforms state-of-the-art deep algorithms in qualitative and quantitative metrics.

Reduces overfitting and improves generalization on the MPI Sintel dataset.

Effectively recovers non scale-invariant quantities in intrinsic image decomposition.

Abstract

We present a new deep supervised learning method for intrinsic decomposition of a single image into its albedo and shading components. Our contributions are based on a new fully convolutional neural network that estimates absolute albedo and shading jointly. Our solution relies on a single end-to-end deep sequence of residual blocks and a perceptually-motivated metric formed by two adversarially trained discriminators. As opposed to classical intrinsic image decomposition work, it is fully data-driven, hence does not require any physical priors like shading smoothness or albedo sparsity, nor does it rely on geometric information such as depth. Compared to recent deep learning techniques, we simplify the architecture, making it easier to build and train, and constrain it to generate a valid and reversible decomposition. We rediscuss and augment the set of quantitative metrics so as to account for the more challenging recovery of non scale-invariant quantities. We train and demonstrate our architecture on the publicly available MPI Sintel dataset and its intrinsic image decomposition, show attenuated overfitting issues and discuss generalizability to other data. Results show that our work outperforms the state of the art deep algorithms both on the qualitative and quantitative aspect.