My camera can see through fences: A deep learning approach for image de-fencing

TL;DR

This paper introduces a deep learning-based semi-automated method for removing fences from images and videos, leveraging CNNs for fence detection and advanced optimization for image restoration, improving robustness over traditional techniques.

Contribution

It presents a novel semi-automated de-fencing algorithm that uses CNNs for fence detection in dynamic scenes and applies split Bregman optimization for image restoration.

Findings

Outperforms existing lattice detection algorithms on benchmark datasets

Effective fence removal in dynamic scenes with complex fences

Quantitative and qualitative validation of the proposed method

Abstract

In recent times, the availability of inexpensive image capturing devices such as smartphones/tablets has led to an exponential increase in the number of images/videos captured. However, sometimes the amateur photographer is hindered by fences in the scene which have to be removed after the image has been captured. Conventional approaches to image de-fencing suffer from inaccurate and non-robust fence detection apart from being limited to processing images of only static occluded scenes. In this paper, we propose a semi-automated de-fencing algorithm using a video of the dynamic scene. We use convolutional neural networks for detecting fence pixels. We provide qualitative as well as quantitative comparison results with existing lattice detection algorithms on the existing PSU NRT data set and a proposed challenging fenced image dataset. The inverse problem of fence removal is solved using split Bregman technique assuming total variation of the de-fenced image as the regularization constraint.