eSkiTB: A Synthetic Event-based Dataset for Tracking Skiers

TL;DR

This paper introduces eSkiTB, a synthetic event-based dataset for ski tracking, demonstrating that event cameras provide superior robustness to clutter compared to RGB, with a new spiking transformer model achieving high accuracy.

Contribution

The paper presents the first controlled event-based ski tracking dataset, enabling comparison with RGB methods, and introduces SDTrack, a spiking transformer that outperforms RGB trackers in cluttered scenes.

Findings

Event-based tracking outperforms RGB in cluttered scenes (+20 IoU points).

SDTrack achieves a mean IoU of 0.711 on eSkiTB.

eSkiTB enables controlled benchmarking for winter-sport tracking.

Abstract

Tracking skiers in RGB broadcast footage is challenging due to motion blur, static overlays, and clutter that obscure the fast-moving athlete. Event cameras, with their asynchronous contrast sensing, offer natural robustness to such artifacts, yet a controlled benchmark for winter-sport tracking has been missing. We introduce event SkiTB (eSkiTB), a synthetic event-based ski tracking dataset generated from SkiTB using direct video-to-event conversion without neural interpolation, enabling an iso-informational comparison between RGB and event modalities. Benchmarking SDTrack (spiking transformer) against STARK (RGB transformer), we find that event-based tracking is substantially resilient to broadcast clutter in scenes dominated by static overlays, achieving 0.685 IoU, outperforming RGB by +20.0 points. Across the dataset, SDTrack attains a mean IoU of 0.711, demonstrating that temporal contrast is a reliable cue for tracking ballistic motion in visually congested environments. eSkiTB establishes the first controlled setting for event-based tracking in winter sports and highlights the promise of event cameras for ski tracking. The dataset and code will be released at https://github.com/eventbasedvision/eSkiTB.