Augmenting Sports Videos with VisCommentator

TL;DR

This paper introduces VisCommentator, a tool that simplifies creating augmented sports videos by combining a systematic design space with machine learning, enabling analysts to efficiently visualize data insights in sports videos.

Contribution

It presents a comprehensive design space for augmented sports videos and a prototype tool that streamlines the creation process using machine learning and visualization recommendations.

Findings

High user satisfaction with VisCommentator.

Participants can quickly reproduce augmented videos.

The system is adaptable to other racket sports.

Abstract

Visualizing data in sports videos is gaining traction in sports analytics, given its ability to communicate insights and explicate player strategies engagingly. However, augmenting sports videos with such data visualizations is challenging, especially for sports analysts, as it requires considerable expertise in video editing. To ease the creation process, we present a design space that characterizes augmented sports videos at an element-level (what the constituents are) and clip-level (how those constituents are organized). We do so by systematically reviewing 233 examples of augmented sports videos collected from TV channels, teams, and leagues. The design space guides selection of data insights and visualizations for various purposes. Informed by the design space and close collaboration with domain experts, we design VisCommentator, a fast prototyping tool, to eases the creation of augmented table tennis videos by leveraging machine learning-based data extractors and design space-based visualization recommendations. With VisCommentator, sports analysts can create an augmented video by selecting the data to visualize instead of manually drawing the graphical marks. Our system can be generalized to other racket sports (e.g., tennis, badminton) once the underlying datasets and models are available. A user study with seven domain experts shows high satisfaction with our system, confirms that the participants can reproduce augmented sports videos in a short period, and provides insightful implications into future improvements and opportunities.