SpectralSplat: Appearance-Disentangled Feed-Forward Gaussian Splatting for Driving Scenes

TL;DR

SpectralSplat is a novel feed-forward Gaussian Splatting method that disentangles appearance from geometry, enabling relighting and appearance transfer in autonomous driving scene reconstructions.

Contribution

It introduces a disentanglement framework using appearance-conditioned streams and a hybrid relighting pipeline within Gaussian Splatting.

Findings

Preserves high-quality scene reconstruction.

Enables controllable appearance transfer.

Supports temporally consistent relighting across sequences.

Abstract

Feed-forward 3D Gaussian Splatting methods have achieved impressive reconstruction quality for autonomous driving scenes, yet they entangle scene geometry with transient appearance properties such as lighting, weather, and time of day. This coupling prevents relighting, appearance transfer, and consistent rendering across multi-traversal data captured under varying environmental conditions. We present SpectralSplat, a method that disentangles appearance from geometry within a feed-forward Gaussian Splatting framework. Our key insight is to factor color prediction into an appearance-agnostic base stream and and appearance-conditioned adapted stream, both produced by a shared MLP conditioned on a global appearance embedding derived from DINOv2 features. To enforce disentanglement, we train with paired observations generated by a hybrid relighting pipeline that combines physics-based intrinsic decomposition with diffusion based generative refinement, and supervise with complementary consistency, reconstruction, cross-appearance, and base color losses. We further introduce an appearance-adaptable temporal history that stores appearance-agnostic features, enabling accumulated Gaussians to be re-rendered under arbitrary target appearances. Experiments demonstrate that SpectralSplat preserves the reconstruction quality of the underlying backbone while enabling controllable appearance transfer and temporally consistent relighting across driving sequences.