ZDySS -- Zero-Shot Dynamic Scene Stylization using Gaussian Splatting

TL;DR

ZDySS is a zero-shot dynamic scene stylization framework that uses Gaussian splatting and feature transfer to achieve consistent, high-quality stylization across space and time without prior style training.

Contribution

The paper introduces ZDySS, a novel zero-shot stylization method for dynamic scenes that leverages Gaussian splatting and feature transfer for improved consistency and generalization.

Findings

Outperforms state-of-the-art baselines in real-world dynamic scene stylization

Achieves high spatio-temporal consistency in stylized videos

Operates without style-specific training, enabling zero-shot generalization

Abstract

Stylizing a dynamic scene based on an exemplar image is critical for various real-world applications, including gaming, filmmaking, and augmented and virtual reality. However, achieving consistent stylization across both spatial and temporal dimensions remains a significant challenge. Most existing methods are designed for static scenes and often require an optimization process for each style image, limiting their adaptability. We introduce ZDySS, a zero-shot stylization framework for dynamic scenes, allowing our model to generalize to previously unseen style images at inference. Our approach employs Gaussian splatting for scene representation, linking each Gaussian to a learned feature vector that renders a feature map for any given view and timestamp. By applying style transfer on the learned feature vectors instead of the rendered feature map, we enhance spatio-temporal consistency across frames. Our method demonstrates superior performance and coherence over state-of-the-art baselines in tests on real-world dynamic scenes, making it a robust solution for practical applications.