SGSST: Scaling Gaussian Splatting StyleTransfer

TL;DR

SGSST is a novel optimization method that enables high-resolution style transfer to pretrained 3D Gaussian splatting scenes, achieving superior visual quality and scalability in neural rendering.

Contribution

It introduces a multiscale loss called SOS that allows style transfer at ultra-high resolutions in 3D scenes, advancing the capabilities of neural rendering techniques.

Findings

Enables style transfer at ultra-high resolutions in 3D scenes

Produces superior visual quality compared to previous methods

Demonstrates effectiveness through qualitative, quantitative, and perceptual evaluations

Abstract

Applying style transfer to a full 3D environment is a challenging task that has seen many developments since the advent of neural rendering. 3D Gaussian splatting (3DGS) has recently pushed further many limits of neural rendering in terms of training speed and reconstruction quality. This work introduces SGSST: Scaling Gaussian Splatting Style Transfer, an optimization-based method to apply style transfer to pretrained 3DGS scenes. We demonstrate that a new multiscale loss based on global neural statistics, that we name SOS for Simultaneously Optimized Scales, enables style transfer to ultra-high resolution 3D scenes. Not only SGSST pioneers 3D scene style transfer at such high image resolutions, it also produces superior visual quality as assessed by thorough qualitative, quantitative and perceptual comparisons.