Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

TL;DR

This paper introduces Differentiable Stereopsis, a novel multi-view stereo method that reconstructs textured 3D meshes from limited noisy inputs by combining traditional stereopsis with differentiable rendering in an end-to-end framework.

Contribution

It presents a new approach that integrates traditional stereopsis with differentiable rendering to produce textured 3D meshes from few views, handling complex shapes and topologies.

Findings

Outperforms traditional multi-view stereo methods in challenging scenes

Produces high-quality textured 3D meshes from noisy camera inputs

Demonstrates effectiveness on diverse real-world objects

Abstract

We propose Differentiable Stereopsis, a multi-view stereo approach that reconstructs shape and texture from few input views and noisy cameras. We pair traditional stereopsis and modern differentiable rendering to build an end-to-end model which predicts textured 3D meshes of objects with varying topologies and shape. We frame stereopsis as an optimization problem and simultaneously update shape and cameras via simple gradient descent. We run an extensive quantitative analysis and compare to traditional multi-view stereo techniques and state-of-the-art learning based methods. We show compelling reconstructions on challenging real-world scenes and for an abundance of object types with complex shape, topology and texture. Project webpage: https://shubham-goel.github.io/ds/