M-SEVIQ: A Multi-band Stereo Event Visual-Inertial Quadruped-based Dataset for Perception under Rapid Motion and Challenging Illumination

TL;DR

M-SEVIQ is a comprehensive multi-band stereo event camera dataset for legged robot perception, capturing diverse real-world scenarios with calibration data to advance perception under rapid motion and challenging lighting.

Contribution

The paper introduces M-SEVIQ, a novel multi-band stereo event camera dataset with extensive calibration, addressing limitations of existing datasets for agile robot perception in challenging environments.

Findings

Over 30 real-world sequences across various conditions

Includes detailed calibration data for sensor fusion

Supports research in perception, sensor fusion, and semantic segmentation

Abstract

Agile locomotion in legged robots poses significant challenges for visual perception. Traditional frame-based cameras often fail in these scenarios for producing blurred images, particularly under low-light conditions. In contrast, event cameras capture changes in brightness asynchronously, offering low latency, high temporal resolution, and high dynamic range. These advantages make them suitable for robust perception during rapid motion and under challenging illumination. However, existing event camera datasets exhibit limitations in stereo configurations and multi-band sensing domains under various illumination conditions. To address this gap, we present M-SEVIQ, a multi-band stereo event visual and inertial quadruped dataset collected using a Unitree Go2 equipped with stereo event cameras, a frame-based camera, an inertial measurement unit (IMU), and joint encoders. This dataset contains more than 30 real-world sequences captured across different velocity levels, illumination wavelengths, and lighting conditions. In addition, comprehensive calibration data, including intrinsic, extrinsic, and temporal alignments, are provided to facilitate accurate sensor fusion and benchmarking. Our M-SEVIQ can be used to support research in agile robot perception, sensor fusion, semantic segmentation and multi-modal vision in challenging environments.