NOCaL: Calibration-Free Semi-Supervised Learning of Odometry and Camera Intrinsics

TL;DR

NOCaL is a semi-supervised learning framework that enables calibration-free odometry and scene understanding for unseen cameras using a hypernetwork trained on diverse existing cameras.

Contribution

It introduces a novel scene-rendering hypernetwork that adapts to new cameras with minimal supervision, eliminating the need for calibration.

Findings

Achieves calibration-free odometry on rendered and real images.

Enables novel view synthesis without camera calibration.

Demonstrates effective adaptation to unseen camera geometries.

Abstract

There are a multitude of emerging imaging technologies that could benefit robotics. However the need for bespoke models, calibration and low-level processing represents a key barrier to their adoption. In this work we present NOCaL, Neural odometry and Calibration using Light fields, a semi-supervised learning architecture capable of interpreting previously unseen cameras without calibration. NOCaL learns to estimate camera parameters, relative pose, and scene appearance. It employs a scene-rendering hypernetwork pretrained on a large number of existing cameras and scenes, and adapts to previously unseen cameras using a small supervised training set to enforce metric scale. We demonstrate NOCaL on rendered and captured imagery using conventional cameras, demonstrating calibration-free odometry and novel view synthesis. This work represents a key step toward automating the interpretation of general camera geometries and emerging imaging technologies.