Gaussian Splatting in Style

TL;DR

This paper introduces a real-time 3D scene stylization method using Gaussian splatting that maintains view consistency and generalizes across styles, outperforming previous approaches in quality and speed.

Contribution

The authors propose a novel 3D Gaussian splatting-based architecture for stylization that enables fast, high-quality, multi-view consistent results with style generalization during testing.

Findings

Achieves state-of-the-art stylization quality on real-world data.

Provides geometric consistency and fast training and rendering.

Enables practical applications like AR and VR.

Abstract

3D scene stylization extends the work of neural style transfer to 3D. A vital challenge in this problem is to maintain the uniformity of the stylized appearance across multiple views. A vast majority of the previous works achieve this by training a 3D model for every stylized image and a set of multi-view images. In contrast, we propose a novel architecture trained on a collection of style images that, at test time, produces real time high-quality stylized novel views. We choose the underlying 3D scene representation for our model as 3D Gaussian splatting. We take the 3D Gaussians and process them using a multi-resolution hash grid and a tiny MLP to obtain stylized views. The MLP is conditioned on different style codes for generalization to different styles during test time. The explicit nature of 3D Gaussians gives us inherent advantages over NeRF-based methods, including geometric consistency and a fast training and rendering regime. This enables our method to be useful for various practical use cases, such as augmented or virtual reality. We demonstrate that our method achieves state-of-the-art performance with superior visual quality on various indoor and outdoor real-world data.