SD-6DoF-ICLK: Sparse and Deep Inverse Compositional Lucas-Kanade Algorithm on SE(3)

TL;DR

This paper presents SD-6DoF-ICLK, a learning-based sparse depth and image alignment algorithm on SE(3) that improves robustness and speed for visual odometry and SLAM applications.

Contribution

It introduces a novel deep inverse compositional Lucas-Kanade method utilizing sparse depth for efficient 6DoF pose estimation on SE(3).

Findings

Runs at 145 ms per image pair at 752x480 resolution.

Outperforms classical sparse 6DoF-ICLK algorithms.

Suitable for robust image alignment in severe conditions.

Abstract

This paper introduces SD-6DoF-ICLK, a learning-based Inverse Compositional Lucas-Kanade (ICLK) pipeline that uses sparse depth information to optimize the relative pose that best aligns two images on SE(3). To compute this six Degrees-of-Freedom (DoF) relative transformation, the proposed formulation requires only sparse depth information in one of the images, which is often the only available depth source in visual-inertial odometry or Simultaneous Localization and Mapping (SLAM) pipelines. In an optional subsequent step, the framework further refines feature locations and the relative pose using individual feature alignment and bundle adjustment for pose and structure re-alignment. The resulting sparse point correspondences with subpixel-accuracy and refined relative pose can be used for depth map generation, or the image alignment module can be embedded in an odometry or mapping framework. Experiments with rendered imagery show that the forward SD-6DoF-ICLK runs at 145 ms per image pair with a resolution of 752 x 480 pixels each, and vastly outperforms the classical, sparse 6DoF-ICLK algorithm, making it the ideal framework for robust image alignment under severe conditions.