Accurate and efficient video de-fencing using convolutional neural networks and temporal information

TL;DR

This paper introduces a novel de-fencing method combining CNN-based segmentation and optical flow recovery, significantly improving accuracy and robustness in removing fences from videos and images.

Contribution

The paper presents a new approach that enhances fence segmentation accuracy and recovery robustness, outperforming existing methods in complex scenarios.

Findings

Achieves state-of-the-art fence segmentation accuracy.

Produces plausible de-fenced videos and images.

Performs well on diverse and complex datasets.

Abstract

De-fencing is to eliminate the captured fence on an image or a video, providing a clear view of the scene. It has been applied for many purposes including assisting photographers and improving the performance of computer vision algorithms such as object detection and recognition. However, the state-of-the-art de-fencing methods have limited performance caused by the difficulty of fence segmentation and also suffer from the motion of the camera or objects. To overcome these problems, we propose a novel method consisting of segmentation using convolutional neural networks and a fast/robust recovery algorithm. The segmentation algorithm using convolutional neural network achieves significant improvement in the accuracy of fence segmentation. The recovery algorithm using optical flow produces plausible de-fenced images and videos. The proposed method is experimented on both our diverse and complex dataset and publicly available datasets. The experimental results demonstrate that the proposed method achieves the state-of-the-art performance for both segmentation and content recovery.