RenderFlow: Single-Step Neural Rendering via Flow Matching

TL;DR

RenderFlow is a deterministic, single-step neural rendering framework that achieves near real-time photorealistic images by using flow matching, significantly reducing latency compared to diffusion-based methods.

Contribution

The paper introduces RenderFlow, a novel flow matching-based neural rendering method that is deterministic, fast, and improves physical plausibility with sparse keyframe guidance.

Findings

RenderFlow achieves near real-time rendering with high photorealism.

Incorporating sparse keyframes improves physical accuracy and visual quality.

The framework extends to inverse rendering tasks like intrinsic decomposition.

Abstract

Conventional physically based rendering (PBR) pipelines generate photorealistic images through computationally intensive light transport simulations. Although recent deep learning approaches leverage diffusion model priors with geometry buffers (G-buffers) to produce visually compelling results without explicit scene geometry or light simulation, they remain constrained by two major limitations. First, the iterative nature of the diffusion process introduces substantial latency. Second, the inherent stochasticity of these generative models compromises physical accuracy and temporal consistency. In response to these challenges, we propose a novel, end-to-end, deterministic, single-step neural rendering framework, RenderFlow, built upon a flow matching paradigm. To further strengthen both rendering quality and generalization, we propose an efficient and effective module for sparse keyframe guidance. Our method significantly accelerates the rendering process and, by optionally incorporating sparsely rendered keyframes as guidance, enhances both the physical plausibility and overall visual quality of the output. The resulting pipeline achieves near real-time performance with photorealistic rendering quality, effectively bridging the gap between the efficiency of modern generative models and the precision of traditional physically based rendering. Furthermore, we demonstrate the versatility of our framework by introducing a lightweight, adapter-based module that efficiently repurposes the pretrained forward model for the inverse rendering task of intrinsic decomposition.