Jersey Number Recognition using Keyframe Identification from Low-Resolution Broadcast Videos

TL;DR

This paper introduces a robust method for jersey number recognition in low-resolution soccer videos by identifying keyframes with high-level information and modeling spatio-temporal context, significantly improving accuracy.

Contribution

The paper proposes a novel keyframe identification module combined with a spatio-temporal network and multi-task loss for improved jersey number detection in challenging real-world videos.

Findings

Achieved over 37% accuracy improvement on test sets with domain gaps.

Demonstrated effectiveness of keyframe selection in low-resolution, occluded videos.

Validated approach on SoccerNet dataset with significant accuracy gains.

Abstract

Player identification is a crucial component in vision-driven soccer analytics, enabling various downstream tasks such as player assessment, in-game analysis, and broadcast production. However, automatically detecting jersey numbers from player tracklets in videos presents challenges due to motion blur, low resolution, distortions, and occlusions. Existing methods, utilizing Spatial Transformer Networks, CNNs, and Vision Transformers, have shown success in image data but struggle with real-world video data, where jersey numbers are not visible in most of the frames. Hence, identifying frames that contain the jersey number is a key sub-problem to tackle. To address these issues, we propose a robust keyframe identification module that extracts frames containing essential high-level information about the jersey number. A spatio-temporal network is then employed to model spatial and temporal context and predict the probabilities of jersey numbers in the video. Additionally, we adopt a multi-task loss function to predict the probability distribution of each digit separately. Extensive evaluations on the SoccerNet dataset demonstrate that incorporating our proposed keyframe identification module results in a significant 37.81% and 37.70% increase in the accuracies of 2 different test sets with domain gaps. These results highlight the effectiveness and importance of our approach in tackling the challenges of automatic jersey number detection in sports videos.