Joint Epipolar Tracking (JET): Simultaneous optimization of epipolar geometry and feature correspondences

TL;DR

The paper introduces JET, a novel sparse direct method that jointly optimizes epipolar geometry and feature correspondences by directly utilizing image intensities, improving pose estimation accuracy and efficiency.

Contribution

It proposes a new joint optimization approach that combines feature correspondence and pose estimation into a single framework considering image intensities and prior motion information.

Findings

Outperforms classical RPE optimization on synthetic and KITTI datasets.

Runs in real-time on a single CPU thread.

Effectively incorporates Bayesian motion priors.

Abstract

Traditionally, pose estimation is considered as a two step problem. First, feature correspondences are determined by direct comparison of image patches, or by associating feature descriptors. In a second step, the relative pose and the coordinates of corresponding points are estimated, most often by minimizing the reprojection error (RPE). RPE optimization is based on a loss function that is merely aware of the feature pixel positions but not of the underlying image intensities. In this paper, we propose a sparse direct method which introduces a loss function that allows to simultaneously optimize the unscaled relative pose, as well as the set of feature correspondences directly considering the image intensity values. Furthermore, we show how to integrate statistical prior information on the motion into the optimization process. This constructive inclusion of a Bayesian bias term is particularly efficient in application cases with a strongly predictable (short term) dynamic, e.g. in a driving scenario. In our experiments, we demonstrate that the JET algorithm we propose outperforms the classical reprojection error optimization on two synthetic datasets and on the KITTI dataset. The JET algorithm runs in real-time on a single CPU thread.