See Without Decoding: Motion-Vector-Based Tracking in Compressed Video

TL;DR

This paper introduces a lightweight tracking method that operates directly on compressed video data, significantly speeding up processing with minimal accuracy loss, enabling real-time large-scale monitoring.

Contribution

It presents a novel compressed-domain tracking model that bypasses full decoding, leveraging motion vectors for efficient real-time object tracking.

Findings

Achieves up to 3.7x speed-up over RGB decoding methods.

Only 4% mAP@0.5 drop compared to RGB baseline.

Effective for real-time analytics in large monitoring systems.

Abstract

We propose a lightweight compressed-domain tracking model that operates directly on video streams, without requiring full RGB video decoding. Using motion vectors and transform coefficients from compressed data, our deep model propagates object bounding boxes across frames, achieving a computational speed-up of order up to 3.7 with only a slight 4% mAP@0.5 drop vs RGB baseline on MOTS15/17/20 datasets. These results highlight codec-domain motion modeling efficiency for real-time analytics in large monitoring systems.