Event-based Mosaicing Bundle Adjustment

TL;DR

This paper introduces EMBA, a novel event-based mosaicing bundle adjustment method that efficiently refines camera orientations and scene maps for rotating event cameras, achieving high-quality panoramas without image conversion.

Contribution

First to exploit block-diagonal sparsity in event-based camera bundle adjustment, enabling faster optimization without converting events into images.

Findings

50% reduction in photometric error

Effective on synthetic and real datasets

Produces high-resolution panoramas in the wild

Abstract

We tackle the problem of mosaicing bundle adjustment (i.e., simultaneous refinement of camera orientations and scene map) for a purely rotating event camera. We formulate the problem as a regularized non-linear least squares optimization. The objective function is defined using the linearized event generation model in the camera orientations and the panoramic gradient map of the scene. We show that this BA optimization has an exploitable block-diagonal sparsity structure, so that the problem can be solved efficiently. To the best of our knowledge, this is the first work to leverage such sparsity to speed up the optimization in the context of event-based cameras, without the need to convert events into image-like representations. We evaluate our method, called EMBA, on both synthetic and real-world datasets to show its effectiveness (50% photometric error decrease), yielding results of unprecedented quality. In addition, we demonstrate EMBA using high spatial resolution event cameras, yielding delicate panoramas in the wild, even without an initial map. Project page: https://github.com/tub-rip/emba