Am I a Baller? Basketball Performance Assessment from First-Person Videos

TL;DR

This paper introduces a novel method for evaluating basketball players' performance from first-person videos using deep learning, addressing subjectivity and camera movement challenges to produce personalized assessments.

Contribution

The paper presents a new approach combining convolutional LSTM networks and Gaussian mixture models to assess basketball performance from first-person videos, tailored to evaluator preferences.

Findings

Accurately assesses player performance in real-world games.

Learns to identify key basketball events impacting performance.

Works despite camera movement and subjective evaluation criteria.

Abstract

This paper presents a method to assess a basketball player's performance from his/her first-person video. A key challenge lies in the fact that the evaluation metric is highly subjective and specific to a particular evaluator. We leverage the first-person camera to address this challenge. The spatiotemporal visual semantics provided by a first-person view allows us to reason about the camera wearer's actions while he/she is participating in an unscripted basketball game. Our method takes a player's first-person video and provides a player's performance measure that is specific to an evaluator's preference. To achieve this goal, we first use a convolutional LSTM network to detect atomic basketball events from first-person videos. Our network's ability to zoom-in to the salient regions addresses the issue of a severe camera wearer's head movement in first-person videos. The detected atomic events are then passed through the Gaussian mixtures to construct a highly non-linear visual spatiotemporal basketball assessment feature. Finally, we use this feature to learn a basketball assessment model from pairs of labeled first-person basketball videos, for which a basketball expert indicates, which of the two players is better. We demonstrate that despite not knowing the basketball evaluator's criterion, our model learns to accurately assess the players in real-world games. Furthermore, our model can also discover basketball events that contribute positively and negatively to a player's performance.