SPIN: A High Speed, High Resolution Vision Dataset for Tracking and Action Recognition in Ping Pong

TL;DR

The paper introduces SPIN, a high-resolution, high frame rate stereo video dataset for ping pong, enabling advanced tracking and action recognition with baseline models and insights into ball trajectories.

Contribution

It provides the first large-scale ping pong dataset with multi-task annotations and baseline models capable of real-time inference at 150 fps.

Findings

Baseline models achieve real-time inference at 150 fps.

Multi-task training improves tracking and recognition performance.

Derived ball trajectories reveal properties beyond physics models.

Abstract

We introduce a new high resolution, high frame rate stereo video dataset, which we call SPIN, for tracking and action recognition in the game of ping pong. The corpus consists of ping pong play with three main annotation streams that can be used to learn tracking and action recognition models -- tracking of the ping pong ball and poses of humans in the videos and the spin of the ball being hit by humans. The training corpus consists of 53 hours of data with labels derived from previous models in a semi-supervised method. The testing corpus contains 1 hour of data with the same information, except that crowd compute was used to obtain human annotations of the ball position, from which ball spin has been derived. Along with the dataset we introduce several baseline models that were trained on this data. The models were specifically chosen to be able to perform inference at the same rate as the images are generated -- specifically 150 fps. We explore the advantages of multi-task training on this data, and also show interesting properties of ping pong ball trajectories that are derived from our observational data, rather than from prior physics models. To our knowledge this is the first large scale dataset of ping pong; we offer it to the community as a rich dataset that can be used for a large variety of machine learning and vision tasks such as tracking, pose estimation, semi-supervised and unsupervised learning and generative modeling.