Game State and Spatio-temporal Action Detection in Soccer using Graph Neural Networks and 3D Convolutional Networks

TL;DR

This paper introduces a novel approach combining Graph Neural Networks and 3D CNNs to improve spatio-temporal action detection in soccer by integrating game state information, addressing the limitations of purely visual methods.

Contribution

It presents an end-to-end model that fuses visual data with contextual game state information using GNNs and 3D CNNs, enhancing action detection accuracy in soccer.

Findings

Improved detection metrics with game state integration

Effective combination of visual and contextual data

Demonstrated end-to-end training benefits

Abstract

Soccer analytics rely on two data sources: the player positions on the pitch and the sequences of events they perform. With around 2000 ball events per game, their precise and exhaustive annotation based on a monocular video stream remains a tedious and costly manual task. While state-of-the-art spatio-temporal action detection methods show promise for automating this task, they lack contextual understanding of the game. Assuming professional players' behaviors are interdependent, we hypothesize that incorporating surrounding players' information such as positions, velocity and team membership can enhance purely visual predictions. We propose a spatio-temporal action detection approach that combines visual and game state information via Graph Neural Networks trained end-to-end with state-of-the-art 3D CNNs, demonstrating improved metrics through game state integration.