N-SfC: Robust and Fast Shape Estimation from Caustic Images

TL;DR

This paper introduces N-SfC, a learning-based method that significantly improves the speed and robustness of shape reconstruction from caustic images, with practical applications in 3D glass printing quality control.

Contribution

We propose Neural-Shape from Caustics (N-SfC), combining denoising and learned gradient descent to enhance shape reconstruction from caustic images efficiently.

Findings

Outperforms state-of-the-art in speed and accuracy

Reduces computational cost of light transport simulation

Achieves better convergence with fewer iterations

Abstract

This paper deals with the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. The recent Shape from Caustics (SfC) method casts the problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practicability with respect to reconstruction speed and robustness. To address these issues, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension that incorporates two components into the reconstruction pipeline: a denoising module, which alleviates the computational cost of the light transport simulation, and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations. Extensive experiments demonstrate the effectiveness of our neural extensions in the scenario of quality control in 3D glass printing, where we significantly outperform the current state-of-the-art in terms of computational speed and final surface error.