BasketLiDAR: The First LiDAR-Camera Multimodal Dataset for Professional Basketball MOT

TL;DR

This paper introduces BasketLiDAR, a multimodal dataset combining LiDAR and camera data for professional basketball, and proposes a novel real-time multi-object tracking framework that improves accuracy and efficiency in complex, occluded scenarios.

Contribution

It presents the first multimodal LiDAR-camera dataset for basketball and develops a new tracking algorithm that enhances real-time performance and accuracy under occlusion.

Findings

Achieves real-time tracking with LiDAR and camera data

Outperforms camera-only methods in accuracy and occlusion handling

Provides a comprehensive dataset for sports MOT research

Abstract

Real-time 3D trajectory player tracking in sports plays a crucial role in tactical analysis, performance evaluation, and enhancing spectator experience. Traditional systems rely on multi-camera setups, but are constrained by the inherently two-dimensional nature of video data and the need for complex 3D reconstruction processing, making real-time analysis challenging. Basketball, in particular, represents one of the most difficult scenarios in the MOT field, as ten players move rapidly and complexly within a confined court space, with frequent occlusions caused by intense physical contact. To address these challenges, this paper constructs BasketLiDAR, the first multimodal dataset in the sports MOT field that combines LiDAR point clouds with synchronized multi-view camera footage in a professional basketball environment, and proposes a novel MOT framework that simultaneously achieves improved tracking accuracy and reduced computational cost. The BasketLiDAR dataset contains a total of 4,445 frames and 3,105 player IDs, with fully synchronized IDs between three LiDAR sensors and three multi-view cameras. We recorded 5-on-5 and 3-on-3 game data from actual professional basketball players, providing complete 3D positional information and ID annotations for each player. Based on this dataset, we developed a novel MOT algorithm that leverages LiDAR's high-precision 3D spatial information. The proposed method consists of a real-time tracking pipeline using LiDAR alone and a multimodal tracking pipeline that fuses LiDAR and camera data. Experimental results demonstrate that our approach achieves real-time operation, which was difficult with conventional camera-only methods, while achieving superior tracking performance even under occlusion conditions. The dataset is available upon request at: https://sites.google.com/keio.jp/keio-csg/projects/basket-lidar