Judging a video by its bitstream cover

TL;DR

This paper introduces a fast, bitstream-only video classification method that bypasses decompression, achieving high accuracy and speed on a large YouTube dataset, outperforming traditional techniques significantly.

Contribution

The novel approach classifies videos directly from bitstream data, reducing computational costs and improving performance in low-quality videos, validated on a large-scale dataset.

Findings

Over 80% precision, accuracy, and recall achieved.

Operates 15,000 times faster than real-time for 30fps videos.

Outperforms traditional DTW algorithm by six orders of magnitude.

Abstract

Classifying videos into distinct categories, such as Sport and Music Video, is crucial for multimedia understanding and retrieval, especially in an age where an immense volume of video content is constantly being generated. Traditional methods require video decompression to extract pixel-level features like color, texture, and motion, thereby increasing computational and storage demands. Moreover, these methods often suffer from performance degradation in low-quality videos. We present a novel approach that examines only the post-compression bitstream of a video to perform classification, eliminating the need for bitstream. We validate our approach using a custom-built data set comprising over 29,000 YouTube video clips, totaling 6,000 hours and spanning 11 distinct categories. Our preliminary evaluations indicate precision, accuracy, and recall rates well over 80%. The algorithm operates approximately 15,000 times faster than real-time for 30fps videos, outperforming traditional Dynamic Time Warping (DTW) algorithm by six orders of magnitude.