CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields

TL;DR

CoARF is a novel method that enables fine-grained, controllable artistic style transfer in 3D radiance fields, allowing users to specify styles for objects and regions with improved quality.

Contribution

We introduce CoARF, a controllable 3D style transfer algorithm that uses segmentation masks and semantic-aware matching for precise and customizable scene stylization.

Findings

Enables style transfer for specific objects and regions.

Achieves higher style transfer quality with better feature matching.

Provides user-controlled, semantic-aware 3D scene stylization.

Abstract

Creating artistic 3D scenes can be time-consuming and requires specialized knowledge. To address this, recent works such as ARF, use a radiance field-based approach with style constraints to generate 3D scenes that resemble a style image provided by the user. However, these methods lack fine-grained control over the resulting scenes. In this paper, we introduce Controllable Artistic Radiance Fields (CoARF), a novel algorithm for controllable 3D scene stylization. CoARF enables style transfer for specified objects, compositional 3D style transfer and semantic-aware style transfer. We achieve controllability using segmentation masks with different label-dependent loss functions. We also propose a semantic-aware nearest neighbor matching algorithm to improve the style transfer quality. Our extensive experiments demonstrate that CoARF provides user-specified controllability of style transfer and superior style transfer quality with more precise feature matching.