Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis

TL;DR

This paper introduces a synthetic data augmentation pipeline to improve 3D pose estimation of baseball pitchers from broadcast videos, effectively mitigating motion blur effects and enhancing model robustness in real-world conditions.

Contribution

The authors propose a novel augmentation method combined with in-the-wild data to significantly improve pose estimation accuracy under motion blur.

Findings

54.2% reduction in 2D pose estimation loss

36.2% reduction in 3D pose estimation loss

29.2% average improvement on state-of-the-art models

Abstract

Using videos to analyze pitchers in baseball can play a vital role in strategizing and injury prevention. Computer vision-based pose analysis offers a time-efficient and cost-effective approach. However, the use of accessible broadcast videos, with a 30fps framerate, often results in partial body motion blur during fast actions, limiting the performance of existing pose keypoint estimation models. Previous works have primarily relied on fixed backgrounds, assuming minimal motion differences between frames, or utilized multiview data to address this problem. To this end, we propose a synthetic data augmentation pipeline to enhance the model's capability to deal with the pitcher's blurry actions. In addition, we leverage in-the-wild videos to make our model robust under different real-world conditions and camera positions. By carefully optimizing the augmentation parameters, we observed a notable reduction in the loss by 54.2% and 36.2% on the test dataset for 2D and 3D pose estimation respectively. By applying our approach to existing state-of-the-art pose estimators, we demonstrate an average improvement of 29.2%. The findings highlight the effectiveness of our method in mitigating the challenges posed by motion blur, thereby enhancing the overall quality of pose estimation.