A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond

TL;DR

This survey reviews recent deep learning methods for intrinsic image decomposition, highlighting advances beyond Lambertian assumptions towards more physically accurate models and discussing future research directions.

Contribution

It provides a comprehensive overview of deep learning approaches for intrinsic images, classifies methods by priors and models, and discusses future directions with neural rendering techniques.

Findings

Deep learning improves intrinsic image separation accuracy.

Increasing use of physically-principled models beyond Lambertian assumptions.

Insights into future research directions with neural and inverse rendering.

Abstract

Intrinsic imaging or intrinsic image decomposition has traditionally been described as the problem of decomposing an image into two layers: a reflectance, the albedo invariant color of the material; and a shading, produced by the interaction between light and geometry. Deep learning techniques have been broadly applied in recent years to increase the accuracy of those separations. In this survey, we overview those results in context of well-known intrinsic image data sets and relevant metrics used in the literature, discussing their suitability to predict a desirable intrinsic image decomposition. Although the Lambertian assumption is still a foundational basis for many methods, we show that there is increasing awareness on the potential of more sophisticated physically-principled components of the image formation process, that is, optically accurate material models and geometry, and more complete inverse light transport estimations. We classify these methods in terms of the type of decomposition, considering the priors and models used, as well as the learning architecture and methodology driving the decomposition process. We also provide insights about future directions for research, given the recent advances in neural, inverse and differentiable rendering techniques.