Event-based Structure-from-Orbit

TL;DR

This paper introduces eSfO, a method that uses event sensors to reconstruct 3D structures and orbital motion of spinning objects, overcoming occlusion challenges with novel tracking and optimization techniques.

Contribution

The paper presents a novel event feature tracker and a factor graph-based back-end for simultaneous 3D reconstruction and motion estimation of spinning objects from event data.

Findings

Effective tracking of helical feature trajectories.

Accurate estimation of spin rate and rotational axis.

Validated on a new event dataset with ground truth.

Abstract

Event sensors offer high temporal resolution visual sensing, which makes them ideal for perceiving fast visual phenomena without suffering from motion blur. Certain applications in robotics and vision-based navigation require 3D perception of an object undergoing circular or spinning motion in front of a static camera, such as recovering the angular velocity and shape of the object. The setting is equivalent to observing a static object with an orbiting camera. In this paper, we propose event-based structure-from-orbit (eSfO), where the aim is to simultaneously reconstruct the 3D structure of a fast spinning object observed from a static event camera, and recover the equivalent orbital motion of the camera. Our contributions are threefold: since state-of-the-art event feature trackers cannot handle periodic self-occlusion due to the spinning motion, we develop a novel event feature tracker based on spatio-temporal clustering and data association that can better track the helical trajectories of valid features in the event data. The feature tracks are then fed to our novel factor graph-based structure-from-orbit back-end that calculates the orbital motion parameters (e.g., spin rate, relative rotational axis) that minimize the reprojection error. For evaluation, we produce a new event dataset of objects under spinning motion. Comparisons against ground truth indicate the efficacy of eSfO.