A Photometrically Calibrated Benchmark For Monocular Visual Odometry

TL;DR

This paper introduces a new, photometrically calibrated dataset for evaluating monocular visual odometry and SLAM, enabling more accurate assessment of tracking accuracy through accumulated drift analysis.

Contribution

It provides a diverse, real-world dataset with photometric calibration and a novel vignette calibration method, along with an evaluation of existing algorithms on this dataset.

Findings

Existing methods show varying performance depending on image resolution and camera FOV.

Photometric calibration improves the reliability of visual odometry evaluation.

The dataset enables drift-based accuracy assessment without full ground truth.

Abstract

We present a dataset for evaluating the tracking accuracy of monocular visual odometry and SLAM methods. It contains 50 real-world sequences comprising more than 100 minutes of video, recorded across dozens of different environments -- ranging from narrow indoor corridors to wide outdoor scenes. All sequences contain mostly exploring camera motion, starting and ending at the same position. This allows to evaluate tracking accuracy via the accumulated drift from start to end, without requiring ground truth for the full sequence. In contrast to existing datasets, all sequences are photometrically calibrated. We provide exposure times for each frame as reported by the sensor, the camera response function, and dense lens attenuation factors. We also propose a novel, simple approach to non-parametric vignette calibration, which requires minimal set-up and is easy to reproduce. Finally, we thoroughly evaluate two existing methods (ORB-SLAM and DSO) on the dataset, including an analysis of the effect of image resolution, camera field of view, and the camera motion direction.