CalibAnyView: Beyond Single-View Camera Calibration in the Wild

TL;DR

CalibAnyView introduces a multi-view learning framework that models cross-view geometric consistency for camera calibration in real-world, uncontrolled environments, outperforming existing methods.

Contribution

It presents a unified multi-view calibration approach with a new dataset and transformer-based dense perspective prediction, enhancing robustness and accuracy.

Findings

Outperforms state-of-the-art calibration methods

Achieves robustness in single-view scenarios

Improves calibration accuracy with multi-view inference

Abstract

Camera calibration is a fundamental prerequisite for reliable geometric perception, yet classical approaches rely on controlled acquisition setups that are impractical for in-the-wild imagery. Recent learning-based methods have shown promising results for single-view calibration, but inherently neglect geometric consistency across multiple views. We introduce CalibAnyView, a unified formulation that supports an arbitrary number of input views ($N \geq 1$) by explicitly modeling cross-view geometric consistency. To facilitate this, we construct a large-scale multi-view video dataset covering diverse real-world scenarios, including multiple camera models, dynamic scenes, realistic motion trajectories, and heterogeneous lens distortions. Building on this dataset, we develop a multi-view transformer that predicts dense perspective fields, which are further integrated into a geometric optimization framework to jointly estimate camera intrinsics and gravity direction. Extensive experiments demonstrate that CalibAnyView consistently outperforms state-of-the-art methods, achieves strong robustness under single-view settings, and further improves with multi-view inference, providing a reliable foundation for downstream tasks such as 3D reconstruction and robotic perception in the wild.