# Neural Inverse Rendering of an Indoor Scene from a Single Image

## TL;DR

This paper introduces a novel learning-based method for inverse rendering of indoor scenes from a single image, jointly estimating reflectance, geometry, and lighting with a new renderer that captures complex effects.

## Contribution

The paper presents the first joint estimation approach for albedo, normals, and lighting from a single image, utilizing a Residual Appearance Renderer for realistic synthesis and self-supervised learning.

## Key findings

- Outperforms existing methods in scene attribute estimation
- Uses a large synthetic dataset for training
- Effectively models complex appearance effects

## Abstract

Inverse rendering aims to estimate physical attributes of a scene, e.g., reflectance, geometry, and lighting, from image(s). Inverse rendering has been studied primarily for single objects or with methods that solve for only one of the scene attributes. We propose the first learning-based approach that jointly estimates albedo, normals, and lighting of an indoor scene from a single image. Our key contribution is the Residual Appearance Renderer (RAR), which can be trained to synthesize complex appearance effects (e.g., inter-reflection, cast shadows, near-field illumination, and realistic shading), which would be neglected otherwise. This enables us to perform self-supervised learning on real data using a reconstruction loss, based on re-synthesizing the input image from the estimated components. We finetune with real data after pretraining with synthetic data. To this end, we use physically-based rendering to create a large-scale synthetic dataset, which is a significant improvement over prior datasets. Experimental results show that our approach outperforms state-of-the-art methods that estimate one or more scene attributes.