EventAug: Multifaceted Spatio-Temporal Data Augmentation Methods for Event-based Learning

TL;DR

EventAug introduces novel spatio-temporal data augmentation techniques for event-based learning, significantly enhancing model robustness and accuracy across various tasks by enriching motion diversity and object variants.

Contribution

The paper presents a systematic augmentation scheme, EventAug, with methods like MSTI, SSEM, and TSEM to improve event data diversity and model performance.

Findings

Achieved a 4.87% accuracy gain on DVS128 Gesture

Enhanced model robustness to occlusions and varied speeds

Consistently improved performance across different tasks and backbones

Abstract

The event camera has demonstrated significant success across a wide range of areas due to its low time latency and high dynamic range. However, the community faces challenges such as data deficiency and limited diversity, often resulting in over-fitting and inadequate feature learning. Notably, the exploration of data augmentation techniques in the event community remains scarce. This work aims to address this gap by introducing a systematic augmentation scheme named EventAug to enrich spatial-temporal diversity. In particular, we first propose Multi-scale Temporal Integration (MSTI) to diversify the motion speed of objects, then introduce Spatial-salient Event Mask (SSEM) and Temporal-salient Event Mask (TSEM) to enrich object variants. Our EventAug can facilitate models learning with richer motion patterns, object variants and local spatio-temporal relations, thus improving model robustness to varied moving speeds, occlusions, and action disruptions. Experiment results show that our augmentation method consistently yields significant improvements across different tasks and backbones (e.g., a 4.87% accuracy gain on DVS128 Gesture). Our code will be publicly available for this community.