Learning To Score Olympic Events

TL;DR

This paper introduces three deep learning frameworks using 3D CNN features for scoring Olympic sports actions, demonstrating improved accuracy over existing methods and offering insights for feedback and evaluation.

Contribution

It presents novel LSTM and SVR-based models with an efficient training mechanism for small datasets in Olympic action scoring.

Findings

SVR-based models outperform LSTM in accuracy

LSTM models provide better action description for feedback

Significant improvement over previous quality assessment methods

Abstract

Estimating action quality, the process of assigning a "score" to the execution of an action, is crucial in areas such as sports and health care. Unlike action recognition, which has millions of examples to learn from, the action quality datasets that are currently available are small -- typically comprised of only a few hundred samples. This work presents three frameworks for evaluating Olympic sports which utilize spatiotemporal features learned using 3D convolutional neural networks (C3D) and perform score regression with i) SVR, ii) LSTM, and iii) LSTM followed by SVR. An efficient training mechanism for the limited data scenarios is presented for clip-based training with LSTM. The proposed systems show significant improvement over existing quality assessment approaches on the task of predicting scores of Olympic events {diving, vault, figure skating}. While the SVR-based frameworks yield better results, LSTM-based frameworks are more natural for describing an action and can be used for improvement feedback.