Residual path integrals for re-rendering

TL;DR

This paper introduces a novel residual path integral approach for efficient incremental re-rendering of scenes with dynamic objects, significantly speeding up scene updates by focusing on residual energy contributions.

Contribution

It develops importance sampling and path mapping strategies specifically for residual path integrals in dynamic scene re-rendering, extending gradient-domain techniques.

Findings

Achieves speed-ups over previous correlated sampling methods.

Effectively handles scene modifications with minimal re-computation.

Provides new insights into re-rendering and potential for advanced sampling techniques.

Abstract

Conventional rendering techniques are primarily designed and optimized for single-frame rendering. In practical applications, such as scene editing and animation rendering, users frequently encounter scenes where only a small portion is modified between consecutive frames. In this paper, we develop a novel approach to incremental re-rendering of scenes with dynamic objects, where only a small part of a scene moves from one frame to the next. We formulate the difference (or residual) in the image between two frames as a (correlated) light-transport integral which we call the residual path integral. Efficient numerical solution of this integral then involves (1)~devising importance sampling strategies to focus on paths with non-zero residual-transport contributions and (2)~choosing appropriate mappings between the native path spaces of the two frames. We introduce a set of path importance sampling strategies that trace from the moving object(s) which are the sources of residual energy. We explore path mapping strategies that generalize those from gradient-domain path tracing to our importance sampling techniques specially for dynamic scenes. Additionally, our formulation can be applied to material editing as a simpler special case. We demonstrate speed-ups over previous correlated sampling of path differences and over rendering the new frame independently. Our formulation brings new insights into the re-rendering problem and paves the way for devising new types of sampling techniques and path mappings with different trade-offs.