ReStyle3D: Scene-Level Appearance Transfer with Semantic Correspondences

TL;DR

ReStyle3D is a new framework that enables scene-level appearance transfer from a single style image to multi-view real-world scenes, ensuring semantic consistency and multi-view coherence.

Contribution

It introduces a semantic correspondence-based approach for multi-view stylization, combining explicit semantic matching with a learned warp-and-refine process, which is novel in scene-level stylization.

Findings

Outperforms prior methods in structure preservation and style similarity

Achieves high multi-view coherence and photo-realism

Supports applications in interior design and virtual staging

Abstract

We introduce ReStyle3D, a novel framework for scene-level appearance transfer from a single style image to a real-world scene represented by multiple views. The method combines explicit semantic correspondences with multi-view consistency to achieve precise and coherent stylization. Unlike conventional stylization methods that apply a reference style globally, ReStyle3D uses open-vocabulary segmentation to establish dense, instance-level correspondences between the style and real-world images. This ensures that each object is stylized with semantically matched textures. It first transfers the style to a single view using a training-free semantic-attention mechanism in a diffusion model. It then lifts the stylization to additional views via a learned warp-and-refine network guided by monocular depth and pixel-wise correspondences. Experiments show that ReStyle3D consistently outperforms prior methods in structure preservation, perceptual style similarity, and multi-view coherence. User studies further validate its ability to produce photo-realistic, semantically faithful results. Our code, pretrained models, and dataset will be publicly released, to support new applications in interior design, virtual staging, and 3D-consistent stylization.