Spotting Temporally Precise, Fine-Grained Events in Video

TL;DR

This paper introduces E2E-Spot, an end-to-end model for accurately detecting the exact timing of fine-grained events in videos, addressing the limitations of existing methods in global reasoning and local detail detection.

Contribution

The paper presents E2E-Spot, a novel, efficient model for precise event spotting in videos, and provides new annotated datasets for future research in this area.

Findings

E2E-Spot outperforms existing baselines in precise event detection.

The model trains quickly on a single GPU.

New datasets and splits enable future research on fine-grained event spotting.

Abstract

We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur). Precise spotting requires models to reason globally about the full-time scale of actions and locally to identify subtle frame-to-frame appearance and motion differences that identify events during these actions. Surprisingly, we find that top performing solutions to prior video understanding tasks such as action detection and segmentation do not simultaneously meet both requirements. In response, we propose E2E-Spot, a compact, end-to-end model that performs well on the precise spotting task and can be trained quickly on a single GPU. We demonstrate that E2E-Spot significantly outperforms recent baselines adapted from the video action detection, segmentation, and spotting literature to the precise spotting task. Finally, we contribute new annotations and splits to several fine-grained sports action datasets to make these datasets suitable for future work on precise spotting.