Automatic Image De-fencing System

TL;DR

This paper presents an automatic system for removing fences from videos, combining learning-based fence detection with a belief propagation approach for filling occlusions, improving post-processing of fenced videos.

Contribution

It introduces a novel learning-based fence detection method and formulates an optimization framework for effective fence removal in videos.

Findings

Fence detection accuracy is significantly improved using HOG descriptors and SVM classifiers.

The proposed de-fencing algorithm effectively restores occluded background scenes.

Experimental results demonstrate the system's effectiveness on real-world videos.

Abstract

Tourists and Wild-life photographers are often hindered in capturing their cherished images or videos by a fence that limits accessibility to the scene of interest. The situation has been exacerbated by growing concerns of security at public places and a need exists to provide a tool that can be used for post-processing such fenced videos to produce a de-fenced image. There are several challenges in this problem, we identify them as Robust detection of fence/occlusions and Estimating pixel motion of background scenes and Filling in the fence/occlusions by utilizing information in multiple frames of the input video. In this work, we aim to build an automatic post-processing tool that can efficiently rid the input video of occlusion artifacts like fences. Our work is distinguished by two major contributions. The first is the introduction of learning based technique to detect the fences patterns with complicated backgrounds. The second is the formulation of objective function and further minimization through loopy belief propagation to fill-in the fence pixels. We observe that grids of Histogram of oriented gradients descriptor using Support vector machines based classifier significantly outperforms detection accuracy of texels in a lattice. We present results of experiments using several real-world videos to demonstrate the effectiveness of the proposed fence detection and de-fencing algorithm.