Intra-frame Object Tracking by Deblatting

TL;DR

This paper introduces a novel tracking method called Tracking by Deblatting that estimates intra-frame object trajectories from motion blur, enabling more precise localization of fast-moving objects in complex videos.

Contribution

The paper presents a new approach that jointly estimates motion blur and object trajectories, improving intra-frame localization accuracy over traditional tracking methods.

Findings

Outperforms baseline in recall and trajectory accuracy.

Uses a novel Trajectory-IoU metric for evaluation.

Effectively tracks high-speed objects with complex motion.

Abstract

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects elapse non-negligible distance during exposure time of a single frame and therefore their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by standard trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. The trajectory is then estimated by fitting a piecewise quadratic curve, which models physically justifiable trajectories. As a result, tracked objects are precisely localized with higher temporal resolution than by conventional trackers. The proposed TbD tracker was evaluated on a newly created dataset of videos with ground truth obtained by a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms baseline both in recall and trajectory accuracy.