DA4Event: towards bridging the Sim-to-Real Gap for Event Cameras using Domain Adaptation

TL;DR

This paper introduces DA4Event, a domain adaptation approach for event cameras, demonstrating improved generalization from simulated to real data and bridging the sim-to-real gap in event-based vision tasks.

Contribution

It proposes a novel Multi-View DA4E architecture that leverages domain adaptation techniques tailored for event data, enhancing transferability from simulation to real-world scenarios.

Findings

DA methods significantly reduce the sim-to-real gap in event data

MV-DA4E outperforms baseline models in cross-domain tests

Effective domain-invariant feature learning for event cameras

Abstract

Event cameras are novel bio-inspired sensors, which asynchronously capture pixel-level intensity changes in the form of "events". The innovative way they acquire data presents several advantages over standard devices, especially in poor lighting and high-speed motion conditions. However, the novelty of these sensors results in the lack of a large amount of training data capable of fully unlocking their potential. The most common approach implemented by researchers to address this issue is to leverage simulated event data. Yet, this approach comes with an open research question: how well simulated data generalize to real data? To answer this, we propose to exploit, in the event-based context, recent Domain Adaptation (DA) advances in traditional computer vision, showing that DA techniques applied to event data help reduce the sim-to-real gap. To this purpose, we propose a novel architecture, which we call Multi-View DA4E (MV-DA4E), that better exploits the peculiarities of frame-based event representations while also promoting domain invariant characteristics in features. Through extensive experiments, we prove the effectiveness of DA methods and MV-DA4E on N-Caltech101. Moreover, we validate their soundness in a real-world scenario through a cross-domain analysis on the popular RGB-D Object Dataset (ROD), which we extended to the event modality (RGB-E).