D-Rex : Diffusion Rendering for Relightable Expressive Avatars

TL;DR

D-Rex is a novel framework that decouples relighting from avatar modeling, enabling photorealistic, expressive, and relightable full-body avatars with view- and temporally consistent relighting.

Contribution

It introduces a learned image-space relighting method using a diffusion model fine-tuned on paired data, compatible with existing white-light avatar systems.

Findings

Enables high-quality relighting of expressive avatars.

Outperforms physically-based relightable avatar methods.

Maintains facial detail and motion during relighting.

Abstract

We present D-Rex, a person-specific framework for photorealistic, relightable, expressive, and animatable full-body human avatars with free-viewpoint rendering. Existing methods for relightable full-body avatars rely on explicit 3D intrinsic decomposition with analytic reflectance models, which require accurate geometry registration and careful optimization to capture realistic light transport effects. This tight coupling of relighting with avatar modeling has hindered expressiveness: to our knowledge, no existing method demonstrates strong facial animation alongside relighting, limiting applicability in telepresence, gaming, and virtual production. We propose to decouple relighting entirely from avatar modeling by treating it as an image-space post-process: a learned translation from flat-lit, albedo-like renderings to a target HDR illumination. To this end, we leverage the strong generative prior of a pre-trained video diffusion relighting model, fine-tuned via LoRA on paired flat-lit and relit frames captured in a light stage. The flat-lit driving frames are produced by an independent expressive full-body avatar framework trained under white-light conditions, requiring no modification to support relighting, making D-Rex directly applicable to any white-light avatar system. We demonstrate that D-Rex enables view- and temporally consistent relighting while faithfully preserving expressive motion and fine-grained facial detail, outperforming physically-based relightable avatar baselines. Project page is https://vcai.mpi-inf.mpg.de/projects/DRex/