Designing An Illumination-Aware Network for Deep Image Relighting

TL;DR

This paper introduces an Illumination-Aware Network (IAN) that efficiently relights images from a single input by mimicking physical lighting processes and incorporating depth information, outperforming previous methods.

Contribution

The paper proposes a novel IAN architecture with an Illumination-Aware Residual Block and depth-guided encoder, enhancing image relighting accuracy and generalization.

Findings

Outperforms state-of-the-art relighting methods in quality and accuracy

Efficient single-image relighting with high fidelity

Effective use of depth information for geometry-aware relighting

Abstract

Lighting is a determining factor in photography that affects the style, expression of emotion, and even quality of images. Creating or finding satisfying lighting conditions, in reality, is laborious and time-consuming, so it is of great value to develop a technology to manipulate illumination in an image as post-processing. Although previous works have explored techniques based on the physical viewpoint for relighting images, extensive supervisions and prior knowledge are necessary to generate reasonable images, restricting the generalization ability of these works. In contrast, we take the viewpoint of image-to-image translation and implicitly merge ideas of the conventional physical viewpoint. In this paper, we present an Illumination-Aware Network (IAN) which follows the guidance from hierarchical sampling to progressively relight a scene from a single image with high efficiency. In addition, an Illumination-Aware Residual Block (IARB) is designed to approximate the physical rendering process and to extract precise descriptors of light sources for further manipulations. We also introduce a depth-guided geometry encoder for acquiring valuable geometry- and structure-related representations once the depth information is available. Experimental results show that our proposed method produces better quantitative and qualitative relighting results than previous state-of-the-art methods. The code and models are publicly available on https://github.com/NK-CS-ZZL/IAN.