On the Benefits of Visual Stabilization for Frame- and Event-based Perception

TL;DR

This paper proposes a processing-based visual stabilization method for event- and frame-based perception systems to improve feature tracking and ego-motion estimation accuracy, especially under large orientation changes.

Contribution

It introduces a novel stabilization approach that compensates for camera rotation using attitude data, enhancing perception performance without mechanical stabilizers.

Findings

Stabilization improves feature tracking accuracy by 27.37%.

Ego-motion estimation accuracy increases by 34.82%.

Processing time for linear velocity computation reduces by at least 25%.

Abstract

Vision-based perception systems are typically exposed to large orientation changes in different robot applications. In such conditions, their performance might be compromised due to the inherent complexity of processing data captured under challenging motion. Integration of mechanical stabilizers to compensate for the camera rotation is not always possible due to the robot payload constraints. This paper presents a processing-based stabilization approach to compensate the camera's rotational motion both on events and on frames (i.e., images). Assuming that the camera's attitude is available, we evaluate the benefits of stabilization in two perception applications: feature tracking and estimating the translation component of the camera's ego-motion. The validation is performed using synthetic data and sequences from well-known event-based vision datasets. The experiments unveil that stabilization can improve feature tracking and camera ego-motion estimation accuracy in 27.37% and 34.82%, respectively. Concurrently, stabilization can reduce the processing time of computing the camera's linear velocity by at least 25%. Code is available at https://github.com/tub-rip/visual_stabilization