GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures

TL;DR

GAINS is a two-stage inverse rendering framework that uses learning-based priors to improve geometry and material estimation from sparse multi-view captures, enabling better relighting and view synthesis.

Contribution

It introduces a novel two-stage approach leveraging priors for stable inverse rendering under sparse views, enhancing accuracy over existing methods.

Findings

Significantly improves material parameter accuracy.

Enhances relighting quality and novel-view synthesis.

Outperforms state-of-the-art Gaussian-based inverse rendering methods.

Abstract

Recent advances in Gaussian Splatting-based inverse rendering extend Gaussian primitives with shading parameters and physically grounded light transport, enabling high-quality material recovery from dense multi-view captures. However, these methods degrade sharply under sparse-view settings, where limited observations lead to severe ambiguity between geometry, reflectance, and lighting. We introduce GAINS (Gaussian-based Inverse rendering from Sparse multi-view captures), a two-stage inverse rendering framework that leverages learning-based priors to stabilize geometry and material estimation. GAINS first refines geometry using monocular depth/normal and diffusion priors, then employs segmentation, intrinsic image decomposition (IID), and diffusion priors to regularize material recovery. Extensive experiments on synthetic and real-world datasets show that GAINS significantly improves material parameter accuracy, relighting quality, and novel-view synthesis compared to state-of-the-art Gaussian-based inverse rendering methods, especially under sparse-view settings. Project page: https://patrickbail.github.io/gains/