TennisVid2Text: Fine-grained Descriptions for Domain Specific Videos

TL;DR

This paper presents TennisVid2Text, a system for generating detailed, human-like textual descriptions of lawn tennis match videos by analyzing actions and interactions, leveraging a large corpus of human descriptions.

Contribution

It introduces a domain-specific approach for detailed video description in sports, utilizing a large corpus and low-level analysis to improve semantic richness and readability.

Findings

Effective in generating semantically rich descriptions

Addresses both correctness and readability

Evaluated on a new tennis video dataset

Abstract

Automatically describing videos has ever been fascinating. In this work, we attempt to describe videos from a specific domain - broadcast videos of lawn tennis matches. Given a video shot from a tennis match, we intend to generate a textual commentary similar to what a human expert would write on a sports website. Unlike many recent works that focus on generating short captions, we are interested in generating semantically richer descriptions. This demands a detailed low-level analysis of the video content, specially the actions and interactions among subjects. We address this by limiting our domain to the game of lawn tennis. Rich descriptions are generated by leveraging a large corpus of human created descriptions harvested from Internet. We evaluate our method on a newly created tennis video data set. Extensive analysis demonstrate that our approach addresses both semantic correctness as well as readability aspects involved in the task.