Fast Approximate Matching of Cell-Phone Videos for Robust Background Subtraction

TL;DR

This paper introduces a fast, approximate method for matching cell-phone videos to improve background subtraction by leveraging spatio-temporal frame matching and temporal smoothness, validated through simulations and real-world tests.

Contribution

It presents a novel approach to background subtraction from handheld videos by formulating it as a spatio-temporal frame matching problem with theoretical error analysis.

Findings

Efficient matching algorithm exploits temporal smoothness.

Theoretical error bounds are established.

Method outperforms several existing approaches on real videos.

Abstract

We identify a novel instance of the background subtraction problem that focuses on extracting near-field foreground objects captured using handheld cameras. Given two user-generated videos of a scene, one with and the other without the foreground object(s), our goal is to efficiently generate an output video with only the foreground object(s) present in it. We cast this challenge as a spatio-temporal frame matching problem, and propose an efficient solution for it that exploits the temporal smoothness of the video sequences. We present theoretical analyses for the error bounds of our approach, and validate our findings using a detailed set of simulation experiments. Finally, we present the results of our approach tested on multiple real videos captured using handheld cameras, and compare them to several alternate foreground extraction approaches.