MoCap-less Quantitative Evaluation of Ego-Pose Estimation Without Ground Truth Measurements

TL;DR

This paper introduces a camera-only, MoCap-free method for evaluating ego-pose estimation quality in robotics, enabling quick, visual, and quantitative assessment without specialized external equipment.

Contribution

The authors present a novel visual validation tool that assesses ego-pose accuracy using only onboard camera data, simplifying experimental setups for robotic research.

Findings

Effective in detecting sensor flaws and desynchronization

Validated on UAV and terrestrial robot datasets

Provides rapid, quantitative quality assessment

Abstract

The emergence of data-driven approaches for control and planning in robotics have highlighted the need for developing experimental robotic platforms for data collection. However, their implementation is often complex and expensive, in particular for flying and terrestrial robots where the precise estimation of the position requires motion capture devices (MoCap) or Lidar. In order to simplify the use of a robotic platform dedicated to research on a wide range of indoor and outdoor environments, we present a data validation tool for ego-pose estimation that does not require any equipment other than the on-board camera. The method and tool allow a rapid, visual and quantitative evaluation of the quality of ego-pose sensors and are sensitive to different sources of flaws in the acquisition chain, ranging from desynchronization of the sensor flows to misevaluation of the geometric parameters of the robotic platform. Using computer vision, the information from the sensors is used to calculate the motion of a semantic scene point through its projection to the 2D image space of the on-board camera. The deviations of these keypoints from references created with a semi-automatic tool allow rapid and simple quality assessment of the data collected on the platform. To demonstrate the performance of our method, we evaluate it on two challenging standard UAV datasets as well as one dataset taken from a terrestrial robot.