Leveraging Contextual Cues for Generating Basketball Highlights

TL;DR

This paper presents a novel approach for automatic basketball highlight generation that incorporates environmental contextual cues, resulting in highlights comparable to professional broadcaster edits.

Contribution

It introduces a new dataset with excitement annotations and explores the use of environmental cues, advancing automatic sports highlight generation beyond visual and audio features.

Findings

Highlights are comparable to ESPN's for study participants.

Environmental cues significantly improve highlight quality.

The approach outperforms previous methods using only visual and audio cues.

Abstract

The massive growth of sports videos has resulted in a need for automatic generation of sports highlights that are comparable in quality to the hand-edited highlights produced by broadcasters such as ESPN. Unlike previous works that mostly use audio-visual cues derived from the video, we propose an approach that additionally leverages contextual cues derived from the environment that the game is being played in. The contextual cues provide information about the excitement levels in the game, which can be ranked and selected to automatically produce high-quality basketball highlights. We introduce a new dataset of 25 NCAA games along with their play-by-play stats and the ground-truth excitement data for each basket. We explore the informativeness of five different cues derived from the video and from the environment through user studies. Our experiments show that for our study participants, the highlights produced by our system are comparable to the ones produced by ESPN for the same games.