Fine-Grain Annotation of Cricket Videos

TL;DR

This paper presents a weakly-supervised method for fine-grain temporal annotation of cricket videos using online commentaries, enabling detailed action retrieval without manual labeling.

Contribution

It introduces a two-stage approach leveraging commentary for scene segmentation and semantic annotation at a fine temporal scale, a novel contribution in sports video analysis.

Findings

Enables retrieval of short, specific actions from hours of cricket videos.

Produces large, automatically labeled datasets for machine learning.

Achieves fine-grained semantic annotation without manual effort.

Abstract

The recognition of human activities is one of the key problems in video understanding. Action recognition is challenging even for specific categories of videos, such as sports, that contain only a small set of actions. Interestingly, sports videos are accompanied by detailed commentaries available online, which could be used to perform action annotation in a weakly-supervised setting. For the specific case of Cricket videos, we address the challenge of temporal segmentation and annotation of ctions with semantic descriptions. Our solution consists of two stages. In the first stage, the video is segmented into "scenes", by utilizing the scene category information extracted from text-commentary. The second stage consists of classifying video-shots as well as the phrases in the textual description into various categories. The relevant phrases are then suitably mapped to the video-shots. The novel aspect of this work is the fine temporal scale at which semantic information is assigned to the video. As a result of our approach, we enable retrieval of specific actions that last only a few seconds, from several hours of video. This solution yields a large number of labeled exemplars, with no manual effort, that could be used by machine learning algorithms to learn complex actions.