Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation

TL;DR

This paper presents a robust two-stage pipeline for 3D trajectory and spin estimation of table tennis balls from monocular videos, combining real-world 2D supervision with synthetic 3D data to improve accuracy and robustness.

Contribution

The authors introduce a novel two-stage approach that separates perception and 3D uplifting, enabling effective training with real-world 2D data and synthetic 3D data for practical table tennis analysis.

Findings

Achieved high accuracy in 3D trajectory estimation from monocular videos.

Robust to real-world artifacts like missing detections and varying frame rates.

Enabled practical 3D spin and trajectory analysis for table tennis.

Abstract

Obtaining the precise 3D motion of a table tennis ball from standard monocular videos is a challenging problem, as existing methods trained on synthetic data struggle to generalize to the noisy, imperfect ball and table detections of the real world. This is primarily due to the inherent lack of 3D ground truth trajectories and spin annotations for real-world video. To overcome this, we propose a novel two-stage pipeline that divides the problem into a front-end perception task and a back-end 2D-to-3D uplifting task. This separation allows us to train the front-end components with abundant 2D supervision from our newly created TTHQ dataset, while the back-end uplifting network is trained exclusively on physically-correct synthetic data. We specifically re-engineer the uplifting model to be robust to common real-world artifacts, such as missing detections and varying frame rates. By integrating a ball detector and a table keypoint detector, our approach transforms a proof-of-concept uplifting method into a practical, robust, and high-performing end-to-end application for 3D table tennis trajectory and spin analysis.