ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization

TL;DR

ARF-Plus introduces a controllable 3D neural style transfer framework that enables systematic manipulation of perceptual factors like color, style scale, spatial areas, and depth, enhancing 3D scene stylization flexibility.

Contribution

The paper presents ARF-Plus, a novel framework that provides four types of perceptual controls for 3D scene stylization, addressing the lack of controllability in existing radiance fields style transfer methods.

Findings

Controls effectively manage perceptual factors in stylization.

Framework supports multiple styles simultaneously.

Results demonstrate high-quality, customizable 3D scene stylization.

Abstract

The radiance fields style transfer is an emerging field that has recently gained popularity as a means of 3D scene stylization, thanks to the outstanding performance of neural radiance fields in 3D reconstruction and view synthesis. We highlight a research gap in radiance fields style transfer, the lack of sufficient perceptual controllability, motivated by the existing concept in the 2D image style transfer. In this paper, we present ARF-Plus, a 3D neural style transfer framework offering manageable control over perceptual factors, to systematically explore the perceptual controllability in 3D scene stylization. Four distinct types of controls - color preservation control, (style pattern) scale control, spatial (selective stylization area) control, and depth enhancement control - are proposed and integrated into this framework. Results from real-world datasets, both quantitative and qualitative, show that the four types of controls in our ARF-Plus framework successfully accomplish their corresponding perceptual controls when stylizing 3D scenes. These techniques work well for individual style inputs as well as for the simultaneous application of multiple styles within a scene. This unlocks a realm of limitless possibilities, allowing customized modifications of stylization effects and flexible merging of the strengths of different styles, ultimately enabling the creation of novel and eye-catching stylistic effects on 3D scenes.