Animated 3DGS Avatars in Diverse Scenes with Consistent Lighting and Shadows

TL;DR

This paper introduces Deep Gaussian Shadow Maps (DGSM) for real-time, volumetric shadow computation and environment relighting of animated 3D Gaussian Splatting avatars in diverse scenes, ensuring consistent lighting and shadows without meshing.

Contribution

The paper presents DGSM and SH-based relighting techniques tailored for 3DGS, enabling real-time, volumetric shadows and environment illumination for animated avatars in static and dynamic scenes.

Findings

DGSM enables real-time volumetric shadow rendering in 3DGS.

SH relighting provides coherent environment illumination for avatars.

Method avoids meshing and offline optimization, ensuring efficiency.

Abstract

We present a method for consistent lighting and shadows when animated 3D Gaussian Splatting (3DGS) avatars interact with 3DGS scenes or with dynamic objects inserted into otherwise static scenes. Our key contribution is Deep Gaussian Shadow Maps (DGSM), a modern analogue of the classical shadow mapping algorithm tailored to the volumetric 3DGS representation. Building on the classic deep shadow mapping idea, we show that 3DGS admits closed form light accumulation along light rays, enabling volumetric shadow computation without meshing. For each estimated light, we tabulate transmittance over concentric radial shells and store them in octahedral atlases, which modern GPUs can sample in real time per query to attenuate affected scene Gaussians and thus cast and receive shadows consistently. To relight moving avatars, we approximate the local environment illumination with HDRI probes represented in a spherical harmonic (SH) basis and apply a fast per Gaussian radiance transfer, avoiding explicit BRDF estimation or offline optimization. We demonstrate environment consistent lighting for avatars from AvatarX and ActorsHQ, composited into ScanNet++, DL3DV, and SuperSplat scenes, and show interactions with inserted objects. Across single and multi avatar settings, DGSM and SH relighting operate fully in the volumetric 3DGS representation, yielding coherent shadows and relighting while avoiding meshing.