De-rendering the World's Revolutionary Artefacts

TL;DR

This paper introduces RADAR, a self-supervised method for decomposing images of symmetric artefacts into shape, material, and lighting, enabling realistic rendering and relighting without explicit supervision.

Contribution

RADAR is the first approach to recover environment illumination and surface materials from single images of real artefacts without explicit 3D or multi-view supervision.

Findings

Effective decomposition of real vase images into shape, material, and lighting.

Enables applications like free-viewpoint rendering and relighting.

Demonstrates superior results on a real vase dataset.

Abstract

Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision. However, many of these assume simplistic material and lighting models. We propose a method, termed RADAR, that can recover environment illumination and surface materials from real single-image collections, relying neither on explicit 3D supervision, nor on multi-view or multi-light images. Specifically, we focus on rotationally symmetric artefacts that exhibit challenging surface properties including specular reflections, such as vases. We introduce a novel self-supervised albedo discriminator, which allows the model to recover plausible albedo without requiring any ground-truth during training. In conjunction with a shape reconstruction module exploiting rotational symmetry, we present an end-to-end learning framework that is able to de-render the world's revolutionary artefacts. We conduct experiments on a real vase dataset and demonstrate compelling decomposition results, allowing for applications including free-viewpoint rendering and relighting.