Monte Carlo Rendering to Diffusion Curves with Differential BEM

TL;DR

This paper introduces a robust method for converting noisy Monte Carlo rendered images into editable vector graphics using a novel differential BEM framework, enabling direct extraction of shading information from 3D geometry.

Contribution

It presents a new stochastic optimization approach with a differential BEM framework for extracting diffusion curves from noisy Monte Carlo samples, handling complex geometries and shading.

Findings

Robust extraction of diffusion curves from noisy Monte Carlo samples.

Effective handling of intersecting diffusion curves and color noise.

Single matrix factorization suffices for gradient computation.

Abstract

We present a method for generating vector graphics, in the form of diffusion curves, directly from noisy samples produced by a Monte Carlo renderer. While generating raster images from 3D geometry via Monte Carlo raytracing is commonplace, there is no corresponding practical approach for robustly and directly extracting editable vector images with shading information from 3D geometry. To fill this gap, we formulate the problem as a stochastic optimization problem over the space of geometries and colors of diffusion curve handles, and solve it with the Levenberg-Marquardt algorithm. At the core of our method is a novel differential boundary element method (BEM) framework that reconstructs colors from diffusion curve handles and computes gradients with respect to their parameters, requiring the expensive matrix factorization only once at the beginning of the optimization. Unlike triangulation-based techniques that require a clean domain decomposition, our method is robust to geometrically challenging scenarios, such as intersecting diffusion curves, and to color noise in the target image, enabling the direct use of noisy Monte Carlo samples without requiring a converged, error-free input image. We demonstrate the robustness and broad applicability of our approach across several test cases. Finally, we highlight several open questions raised by our work, which spans both theory and applications.