Real-time detection and tracking of multiple objects with partial decoding in H.264/AVC bitstream domain

TL;DR

This paper presents a real-time method for detecting and tracking multiple objects in H.264/AVC video streams using probabilistic macroblock filtering and partial decoding, effective even with complex object motion and occlusion.

Contribution

The novel approach combines probabilistic spatiotemporal macroblock filtering with partial decoding to enhance real-time multi-object tracking accuracy in H.264/AVC streams.

Findings

Fast processing time for real-time detection and tracking

Effective handling of articulated and changing objects

Improved trajectory accuracy during occlusion

Abstract

In this paper, we show that we can apply probabilistic spatiotemporal macroblock filtering (PSMF) and partial decoding processes to effectively detect and track multiple objects in real time in H.264|AVC bitstreams with stationary background. Our contribution is that our method cannot only show fast processing time but also handle multiple moving objects that are articulated, changing in size or internally have monotonous color, even though they contain a chaotic set of non-homogeneous motion vectors inside. In addition, our partial decoding process for H.264|AVC bitstreams enables to improve the accuracy of object trajectories and overcome long occlusion by using extracted color information.