Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling
Yuxin Dai, Qi Wang, Jingsen Zhu, Dianbing Xi, Yuchi Huo, Chen Qian,, Ying He
TL;DR
MIRReS introduces a two-stage inverse rendering framework that reconstructs explicit geometry and optimizes material and lighting from multi-view images using multi-bounce path tracing and reservoir sampling, achieving state-of-the-art results.
Contribution
The paper presents a novel explicit geometry-based inverse rendering method that incorporates multi-bounce path tracing and reservoir sampling for improved accuracy and convergence.
Findings
Effective estimation of indirect illumination including shadows and reflections.
State-of-the-art decomposition accuracy demonstrated in complex scenarios.
Enhanced geometry enables practical scene editing and relighting.
Abstract
We present MIRReS, a novel two-stage inverse rendering framework that jointly reconstructs and optimizes the explicit geometry, material, and lighting from multi-view images. Unlike previous methods that rely on implicit irradiance fields or simplified path tracing algorithms, our method extracts an explicit geometry (triangular mesh) in stage one, and introduces a more realistic physically-based inverse rendering model that utilizes multi-bounce path tracing and Monte Carlo integration. By leveraging multi-bounce path tracing, our method effectively estimates indirect illumination, including self-shadowing and internal reflections, which improves the intrinsic decomposition of shape, material, and lighting. Moreover, we incorporate reservoir sampling into our framework to address the noise in Monte Carlo integration, enhancing convergence and facilitating gradient-based optimization…
Peer Reviews
Decision·ICLR 2025 Poster
1. This work adapts well-studied PBR techniques, such as path tracing and reservior sampling, to the domain of inverse rendering. I personally appreciate the adaptation of these computer graphics knowledge to address a more computer vision problem, and the results do show an improvement with these contributions. 2. The most related work to this submission is NVdiffrec-MC. From the reported results, this work achieves a better performance both quantitatively and qualitatively. 3. This paper is we
1. This work relies on a high-quality intialization of geometry, for which the authors chose NeuS2. However, the authors acknowledged that some artifacts may still persist in certain areas, which cannot be fully corrected by the subsequent refinement step. Additionally, they discussed the instability of DMTet for geometry representation, which is used by NVdiffrec-MC. I would be interested to see some visual comparisons between these two geometry representations. 2. As the formulation only consi
1. A novel inverse rendering framework that combines the advantages of neural SDF reconstruction and physically-based differentiable rendering. 2. The paper overall is easy to follow and well organized. 3. High-quality inverse rendering results on both synthetic and real datasets compared to several prior strong baselines.
1. Technical novelties of the paper is very limited. This paper's major contributions are covered by two previous works that are not properly cited. Neural-PBIR, Cheng et al.. ICCV 2023 adopts a similar pipeline to first train a neural SDF to get high-quality geometry and then run PBDR to further optimize geometry, materials and lighting. Neural-PBIR also models multiple bounces and uses unbiased gradient for geometry optimization in the PBDR stage. "Parameter-space ReSTIR for Differentiable an
1.) The paper is very well written and easy to follow. 2.) It contains a very good summary of the current state of the art on Inverse Rendering for Shape, Light and Material reconstruction. 3.) The idea to use Reservoir-based Spatio-Temporal Importance Resampling to reduce the computational load of a global illumination calculation is novel and should definitely be published.
There is no code available to evaluate the method independently. The evaluation and the (very good) comparison with other state-of-the-art methods lack a comparison of the results of the specular reflection. Is this estimate much worse than for the other methods? In the appendix, this is mentioned as a limitation of the procedure.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSeismic Imaging and Inversion Techniques · Hydrocarbon exploration and reservoir analysis · Hydraulic Fracturing and Reservoir Analysis