Indirect Match Highlights Detection with Deep Convolutional Neural Networks

TL;DR

This paper proposes a novel approach to highlight detection in sports videos by analyzing audience behavior with deep 3D CNNs, bypassing traditional gameplay-based methods, and demonstrates promising results on ice-hockey data.

Contribution

It introduces a new paradigm that focuses on audience reactions rather than game actions for highlight detection using deep learning.

Findings

Effective highlight likelihood estimation from audience videos

Outperforms traditional gameplay-based highlight detection methods

Encourages further research in audience-focused sports video analysis

Abstract

Highlights in a sport video are usually referred as actions that stimulate excitement or attract attention of the audience. A big effort is spent in designing techniques which find automatically highlights, in order to automatize the otherwise manual editing process. Most of the state-of-the-art approaches try to solve the problem by training a classifier using the information extracted on the tv-like framing of players playing on the game pitch, learning to detect game actions which are labeled by human observers according to their perception of highlight. Obviously, this is a long and expensive work. In this paper, we reverse the paradigm: instead of looking at the gameplay, inferring what could be exciting for the audience, we directly analyze the audience behavior, which we assume is triggered by events happening during the game. We apply deep 3D Convolutional Neural Network (3D-CNN) to extract visual features from cropped video recordings of the supporters that are attending the event. Outputs of the crops belonging to the same frame are then accumulated to produce a value indicating the Highlight Likelihood (HL) which is then used to discriminate between positive (i.e. when a highlight occurs) and negative samples (i.e. standard play or time-outs). Experimental results on a public dataset of ice-hockey matches demonstrate the effectiveness of our method and promote further research in this new exciting direction.