Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection

TL;DR

This paper introduces a two-stage approach combining fine-tuned action recognition models and transformer-based temporal detection to accurately identify and locate soccer events in videos, achieving state-of-the-art results.

Contribution

It presents a novel combination of high-level semantic features and transformer models for precise soccer event detection in videos, with publicly released features to aid further research.

Findings

Achieved state-of-the-art performance in action spotting and replay grounding.

Developed a transformer-based temporal detection module.

Released soccer embedding features for community use.

Abstract

With rapidly evolving internet technologies and emerging tools, sports related videos generated online are increasing at an unprecedentedly fast pace. To automate sports video editing/highlight generation process, a key task is to precisely recognize and locate the events in the long untrimmed videos. In this tech report, we present a two-stage paradigm to detect what and when events happen in soccer broadcast videos. Specifically, we fine-tune multiple action recognition models on soccer data to extract high-level semantic features, and design a transformer based temporal detection module to locate the target events. This approach achieved the state-of-the-art performance in both two tasks, i.e., action spotting and replay grounding, in the SoccerNet-v2 Challenge, under CVPR 2021 ActivityNet workshop. Our soccer embedding features are released at https://github.com/baidu-research/vidpress-sports. By sharing these features with the broader community, we hope to accelerate the research into soccer video understanding.