Are Minimal Radial Distortion Solvers Really Necessary for Relative Pose Estimation?

TL;DR

This paper demonstrates that simple, non-minimal radial distortion correction methods can achieve comparable accuracy to complex minimal solvers in relative pose estimation, simplifying implementation without sacrificing performance.

Contribution

It compares traditional minimal radial distortion solvers with simpler sampling and neural network-based approaches, showing the latter are sufficient in practice.

Findings

Sampling radial undistortion parameters is effective.

Neural network-based distortion estimation performs well.

Complex minimal solvers are often unnecessary.

Abstract

Estimating the relative pose between two cameras is a fundamental step in many applications such as Structure-from-Motion. The common approach to relative pose estimation is to apply a minimal solver inside a RANSAC loop. Highly efficient solvers exist for pinhole cameras. Yet, (nearly) all cameras exhibit radial distortion. Not modeling radial distortion leads to (significantly) worse results. However, minimal radial distortion solvers are significantly more complex than pinhole solvers, both in terms of run-time and implementation efforts. This paper compares radial distortion solvers with two simple-to-implement approaches that do not use minimal radial distortion solvers: The first approach combines an efficient pinhole solver with sampled radial undistortion parameters, where the sampled parameters are used for undistortion prior to applying the pinhole solver. The second approach uses a state-of-the-art neural network to estimate the distortion parameters rather than sampling them from a set of potential values. Extensive experiments on multiple datasets, and different camera setups, show that complex minimal radial distortion solvers are not necessary in practice. We discuss under which conditions a simple sampling of radial undistortion parameters is preferable over calibrating cameras using a learning-based prior approach. Code and newly created benchmark for relative pose estimation under radial distortion are available at https://github.com/kocurvik/rdnet.