Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles

TL;DR

Styl3R enables instant, high-quality 3D scene stylization from sparse images using a novel architecture that separates structure and appearance, outperforming existing methods in speed and consistency.

Contribution

The paper introduces a fast, feed-forward 3D stylization method with a branched architecture and identity loss pre-training, allowing real-time stylization without dense input images.

Findings

Achieves stylization in less than a second.

Maintains multi-view consistency and scene fidelity.

Outperforms existing methods in quality and efficiency.

Abstract

Stylizing 3D scenes instantly while maintaining multi-view consistency and faithfully resembling a style image remains a significant challenge. Current state-of-the-art 3D stylization methods typically involve computationally intensive test-time optimization to transfer artistic features into a pretrained 3D representation, often requiring dense posed input images. In contrast, leveraging recent advances in feed-forward reconstruction models, we demonstrate a novel approach to achieve direct 3D stylization in less than a second using unposed sparse-view scene images and an arbitrary style image. To address the inherent decoupling between reconstruction and stylization, we introduce a branched architecture that separates structure modeling and appearance shading, effectively preventing stylistic transfer from distorting the underlying 3D scene structure. Furthermore, we adapt an identity loss to facilitate pre-training our stylization model through the novel view synthesis task. This strategy also allows our model to retain its original reconstruction capabilities while being fine-tuned for stylization. Comprehensive evaluations, using both in-domain and out-of-domain datasets, demonstrate that our approach produces high-quality stylized 3D content that achieve a superior blend of style and scene appearance, while also outperforming existing methods in terms of multi-view consistency and efficiency.