MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting

TL;DR

MetaGS introduces a meta-learning based 3D relighting method that generalizes to unseen lighting conditions by learning robust Gaussian geometries and incorporating physical priors, outperforming existing approaches in out-of-distribution scenarios.

Contribution

It proposes a novel meta-learning framework for 3D Gaussian splatting combined with physical priors from the Blinn-Phong model to improve out-of-distribution relighting performance.

Findings

Effective in synthetic and real-world datasets

Supports efficient point-light relighting

Generalizes well to unseen environment lighting maps

Abstract

Out-of-distribution (OOD) 3D relighting requires novel view synthesis under unseen lighting conditions that differ significantly from the observed images. Existing relighting methods, which assume consistent light source distributions between training and testing, often degrade in OOD scenarios. We introduce MetaGS to tackle this challenge from two perspectives. First, we propose a meta-learning approach to train 3D Gaussian splatting, which explicitly promotes learning generalizable Gaussian geometries and appearance attributes across diverse lighting conditions, even with biased training data. Second, we embed fundamental physical priors from the Blinn-Phong reflection model into Gaussian splatting, which enhances the decoupling of shading components and leads to more accurate 3D scene reconstruction. Results on both synthetic and real-world datasets demonstrate the effectiveness of MetaGS in challenging OOD relighting tasks, supporting efficient point-light relighting and generalizing well to unseen environment lighting maps.