Robust Localization with Visual-Inertial Odometry Constraints for Markerless Mobile AR

TL;DR

This paper presents VIO-APR, a novel framework that combines visual-inertial odometry with an absolute pose regressor to improve localization accuracy and stability in markerless mobile AR applications.

Contribution

The paper introduces VIO-APR, a new feedback-based framework that integrates VIO and absolute pose regression to enhance AR localization accuracy and stability.

Findings

VIO-APR improves median position accuracy by up to 36%.

VIO-APR increases high-accuracy frame percentage by up to 112%.

VIO-APR reduces low-accuracy frame predictions significantly.

Abstract

Visual Inertial Odometry (VIO) is an essential component of modern Augmented Reality (AR) applications. However, VIO only tracks the relative pose of the device, leading to drift over time. Absolute pose estimation methods infer the device's absolute pose, but their accuracy depends on the input quality. This paper introduces VIO-APR, a new framework for markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. VIO-APR uses VIO to assess the reliability of the APR and the APR to identify and compensate for VIO drift. This feedback loop results in more accurate positioning and more stable AR experiences. To evaluate VIO-APR, we created a dataset that combines camera images with ARKit's VIO system output for six indoor and outdoor scenes of various scales. Over this dataset, VIO-APR improves the median accuracy of popular APR by up to 36\% in position and 29\% in orientation, increases the percentage of frames in the high ($0.25 m, 2^{\circ}$) accuracy level by up to 112\% and reduces the percentage of frames predicted below the low ($5 m, 10^\circ$) accuracy greatly. We implement VIO-APR into a mobile AR application using Unity to demonstrate its capabilities. VIO-APR results in noticeably more accurate localization and a more stable overall experience.