Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning

TL;DR

Wid3R is a novel neural network that enables 3D reconstruction from wide-angle and 360 imagery without explicit calibration, improving generalization and performance over existing methods.

Contribution

It introduces a camera model token and ray-based representation to support wide field-of-view cameras in a feed-forward neural network.

Findings

Achieves up to +33.67 AUC@30 on Zip-NeRF (fisheye)

Achieves up to +77.33 AUC@30 on Stanford2D3D (360)

Supports 360 imagery without explicit calibration or undistortion.

Abstract

We present Wid3R, a feed-forward neural network for multi-view visual geometry reconstruction that supports wide field-of-view camera models. Unlike existing methods that assume rectified or pinhole inputs, Wid3R directly models wide-angle imagery without explicit calibration or undistortion. Our approach leverages a ray-based representation with spherical harmonics and introduces a novel camera model token to enable distortion-aware reconstruction. To the best of our knowledge, Wid3R is the first multi-frame feed-forward 3D reconstruction method that supports 360 imagery. Moreover, we show that conditioning on diverse camera types improves generalization to 360 scenes and alleviates data sparsity issues. Wid3R achieves significant performance gains, improving AUC@30 by up to +33.67 on Zip-NeRF (fisheye) and +77.33 on Stanford2D3D (360).