Pose Refinement with Joint Optimization of Visual Points and Lines

TL;DR

This paper introduces a novel camera pose refinement method that jointly optimizes visual points and lines, improving localization accuracy in feature-less environments through a new line detection CNN and a combined optimization strategy.

Contribution

It presents a complete pipeline with a new line extraction CNN, a geometric matching strategy, and a joint pose optimization approach for enhanced camera re-localization.

Findings

Improved localization accuracy on open datasets

Effective line detection with the VLSE CNN

Enhanced pose refinement through joint optimization

Abstract

High-precision camera re-localization technology in a pre-established 3D environment map is the basis for many tasks, such as Augmented Reality, Robotics and Autonomous Driving. The point-based visual re-localization approaches are well-developed in recent decades, but are insufficient in some feature-less cases. In this paper, we design a complete pipeline for camera pose refinement with points and lines, which contains the innovatively designed line extracting CNN named VLSE, the line matching and the pose optimization approaches. We adopt a novel line representation and customize a hybrid convolution block based on the Stacked Hourglass network, to detect accurate and stable line features on images. Then we apply a geometric-based strategy to obtain precise 2D-3D line correspondences using epipolar constraint and reprojection filtering. A following point-line joint cost function is constructed to optimize the camera pose with the initial coarse pose from the pure point-based localization. Sufficient experiments are conducted on open datasets, i.e, line extractor on Wireframe and YorkUrban, localization performance on InLoc duc1 and duc2, to confirm the effectiveness of our point-line joint pose optimization method.