DeepSportLab: a Unified Framework for Ball Detection, Player Instance Segmentation and Pose Estimation in Team Sports Scenes

TL;DR

This paper introduces DeepSportLab, a unified deep learning framework that simultaneously detects the ball, segments players, and estimates poses in team sports scenes, improving efficiency and performance over separate models.

Contribution

The paper proposes a novel single-model approach combining part intensity fields and spatial embeddings for multiple tasks in sports scene analysis, addressing occlusion and motion blur issues.

Findings

Achieves comparable performance to state-of-the-art separate models.

Effectively handles occlusion and motion blur in team sports scenes.

Demonstrates versatility across ball detection, segmentation, and pose estimation.

Abstract

This paper presents a unified framework to (i) locate the ball, (ii) predict the pose, and (iii) segment the instance mask of players in team sports scenes. Those problems are of high interest in automated sports analytics, production, and broadcast. A common practice is to individually solve each problem by exploiting universal state-of-the-art models, \eg, Panoptic-DeepLab for player segmentation. In addition to the increased complexity resulting from the multiplication of single-task models, the use of the off-the-shelf models also impedes the performance due to the complexity and specificity of the team sports scenes, such as strong occlusion and motion blur. To circumvent those limitations, our paper proposes to train a single model that simultaneously predicts the ball and the player mask and pose by combining the part intensity fields and the spatial embeddings principles. Part intensity fields provide the ball and player location, as well as player joints location. Spatial embeddings are then exploited to associate player instance pixels to their respective player center, but also to group player joints into skeletons. We demonstrate the effectiveness of the proposed model on the DeepSport basketball dataset, achieving comparable performance to the SoA models addressing each individual task separately.