Mining Automatically Estimated Poses from Video Recordings of Top Athletes

TL;DR

This paper introduces algorithms to automatically extract meaningful pose data from sports videos, enabling analysis of cyclic and phase-based motions without manual annotations, demonstrated on swimming and long jump.

Contribution

It presents novel unsupervised mining algorithms for noisy, annotation-free pose data in sports videos, applicable to cycle-based and phase-based sports.

Findings

Effective extraction of cycle speeds and striking poses in swimming.

Automatic phase partitioning in long jump sequences.

Algorithms applicable to various cycle- and phase-based sports.

Abstract

Human pose detection systems based on state-of-the-art DNNs are on the go to be extended, adapted and re-trained to fit the application domain of specific sports. Therefore, plenty of noisy pose data will soon be available from videos recorded at a regular and frequent basis. This work is among the first to develop mining algorithms that can mine the expected abundance of noisy and annotation-free pose data from video recordings in individual sports. Using swimming as an example of a sport with dominant cyclic motion, we show how to determine unsupervised time-continuous cycle speeds and temporally striking poses as well as measure unsupervised cycle stability over time. Additionally, we use long jump as an example of a sport with a rigid phase-based motion to present a technique to automatically partition the temporally estimated pose sequences into their respective phases. This enables the extraction of performance relevant, pose-based metrics currently used by national professional sports associations. Experimental results prove the effectiveness of our mining algorithms, which can also be applied to other cycle-based or phase-based types of sport.