ROOAD: RELLIS Off-road Odometry Analysis Dataset

TL;DR

The ROOAD dataset offers high-quality off-road visual-inertial data, revealing significant performance gaps of current VIO algorithms in unstructured environments and aiding future localization research.

Contribution

This paper introduces the ROOAD dataset, a new off-road visual-inertial dataset with calibration data, and evaluates state-of-the-art VIO algorithms in off-road conditions.

Findings

VIO algorithms perform 2 to 30 times worse off-road.

OpenVINS has better stability and real-time performance.

VINS-Fusion outperforms OpenVINS in tracking accuracy.

Abstract

The development and implementation of visual-inertial odometry (VIO) has focused on structured environments, but interest in localization in off-road environments is growing. In this paper, we present the RELLIS Off-road Odometry Analysis Dataset (ROOAD) which provides high-quality, time-synchronized off-road monocular visual-inertial data sequences to further the development of related research. We evaluated the dataset on two state-of-the-art VIO algorithms, (1) Open-VINS and (2) VINS-Fusion. Our findings indicate that both algorithms perform 2 to 30 times worse on the ROOAD dataset compared to their performance in structured environments. Furthermore, OpenVINS has better tracking stability and real-time performance than VINS-Fusion in the off-road environment, while VINS-Fusion outperformed OpenVINS in tracking accuracy in several data sequences. Since the camera-IMU calibration tool from Kalibr toolkit is used extensively in this work, we have included several calibration data sequences. Our hand measurements show Kalibr's tool achieved +/-1 degree for orientation error and +/-1 mm at best (x- and y-axis) and +/-10 mm (z-axis) at worse for position error in the camera frame between the camera and IMU. This novel dataset provides a new set of scenarios for researchers to design and test their localization algorithms on, as well as critical insights in the current performance of VIO in off-road environments. ROOAD Dataset: github.com/unmannedlab/ROOAD