SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy

TL;DR

SoccerLens introduces a new benchmark for evaluating soccer video understanding models' ability to ground visual cues, revealing that current models lack true visual grounding despite high classification accuracy.

Contribution

The paper presents SoccerLens, a grounded evaluation benchmark with annotated soccer videos and extended attribution methods to assess visual grounding beyond accuracy.

Findings

Current models achieve over 50% grounding performance only under loose cues.

Models underutilize temporal information in soccer videos.

There is a significant gap between classification accuracy and true visual grounding.

Abstract

Vision-language models (VLMs) have recently shown strong potential in soccer video understanding. However, given the high complexity of soccer videos due to large viewpoint variations, rapid shot transitions, and cluttered scenes, it remains unclear on whether VLMs rely on meaningful visual evidence or exploit spurious correlations and shortcut learning. Existing evaluation protocols focus primarily on classification accuracy and do not assess visual grounding. To address this limitation, we introduce SoccerLens, a benchmark for grounded soccer video understanding. The benchmark contains annotated video segments spanning $13$ common soccer events, with structured visual cues organized into three levels of semantic relevance. We further extend the attribution method of Chefer [arXiv:2103.15679] to jointly model spatial and temporal attention, and introduce evaluation metrics that measure whether model attention aligns with annotated cues or drifts toward spurious regions. Our evaluation of state-of-the-art soccer VLMs shows that, despite strong classification accuracy, current models fail to exceed $50\%$ grounding performance even under the loosest cue definitions and consistently underutilize temporal information. These results reveal a substantial gap between predictive performance and true visual grounding, highlighting the need for grounded evaluation in complex spatio-temporal domains such as soccer.