Sampling Neural Radiance Fields for Refractive Objects

TL;DR

This paper introduces a NeRF-based framework for synthesizing novel views of refractive objects by modeling curved light paths in heterogeneous scenes with varying refractive indices, improving rendering quality.

Contribution

It extends NeRF sampling techniques to handle curved paths caused by refraction, reconstructs scene refractive indices from silhouettes, and demonstrates superior rendering results.

Findings

Outperforms state-of-the-art methods quantitatively and qualitatively.

Achieves better perceptual similarity metrics.

Improves rendering quality on synthetic and real scenes.

Abstract

Recently, differentiable volume rendering in neural radiance fields (NeRF) has gained a lot of popularity, and its variants have attained many impressive results. However, existing methods usually assume the scene is a homogeneous volume so that a ray is cast along the straight path. In this work, the scene is instead a heterogeneous volume with a piecewise-constant refractive index, where the path will be curved if it intersects the different refractive indices. For novel view synthesis of refractive objects, our NeRF-based framework aims to optimize the radiance fields of bounded volume and boundary from multi-view posed images with refractive object silhouettes. To tackle this challenging problem, the refractive index of a scene is reconstructed from silhouettes. Given the refractive index, we extend the stratified and hierarchical sampling techniques in NeRF to allow drawing samples along a curved path tracked by the Eikonal equation. The results indicate that our framework outperforms the state-of-the-art method both quantitatively and qualitatively, demonstrating better performance on the perceptual similarity metric and an apparent improvement in the rendering quality on several synthetic and real scenes.