Event-based visual place recognition with ensembles of temporal windows

TL;DR

This paper introduces an ensemble approach for event-based visual place recognition that combines multiple temporal window lengths, improving accuracy and efficiency, demonstrated on a new dataset with significant performance gains over single-window methods.

Contribution

The paper presents a novel ensemble scheme for combining varying temporal windows in event camera processing, including an efficient approximate version, and demonstrates its effectiveness on a new VPR dataset.

Findings

Ensemble schemes outperform single-window baselines in VPR accuracy.

The approximate ensemble achieves computational efficiency with minimal performance loss.

The approach is robust across different feature extraction methods.

Abstract

Event cameras are bio-inspired sensors capable of providing a continuous stream of events with low latency and high dynamic range. As a single event only carries limited information about the brightness change at a particular pixel, events are commonly accumulated into spatio-temporal windows for further processing. However, the optimal window length varies depending on the scene, camera motion, the task being performed, and other factors. In this research, we develop a novel ensemble-based scheme for combining temporal windows of varying lengths that are processed in parallel. For applications where the increased computational requirements of this approach are not practical, we also introduce a new "approximate" ensemble scheme that achieves significant computational efficiencies without unduly compromising the original performance gains provided by the ensemble approach. We demonstrate our ensemble scheme on the visual place recognition (VPR) task, introducing a new Brisbane-Event-VPR dataset with annotated recordings captured using a DAVIS346 color event camera. We show that our proposed ensemble scheme significantly outperforms all the single-window baselines and conventional model-based ensembles, irrespective of the image reconstruction and feature extraction methods used in the VPR pipeline, and evaluate which ensemble combination technique performs best. These results demonstrate the significant benefits of ensemble schemes for event camera processing in the VPR domain and may have relevance to other related processes, including feature tracking, visual-inertial odometry, and steering prediction in driving.