Automatic Objects Removal for Scene Completion

TL;DR

This paper presents a structure-based image completion method for automatic object removal that infers scene structure and synthesizes textures to produce realistic, consistent scene content, aiding large-scale scene reconstruction.

Contribution

It introduces a novel structure-guided image completion algorithm that effectively removes objects and reconstructs scenes with consistent structure and texture.

Findings

Effective object removal across various scene types

Produces visually plausible and structurally consistent results

Suitable for large-scale scene reconstruction tasks

Abstract

With the explosive growth of web-based cameras and mobile devices, billions of photographs are uploaded to the internet. We can trivially collect a huge number of photo streams for various goals, such as 3D scene reconstruction and other big data applications. However, this is not an easy task due to the fact the retrieved photos are neither aligned nor calibrated. Furthermore, with the occlusion of unexpected foreground objects like people, vehicles, it is even more challenging to find feature correspondences and reconstruct realistic scenes. In this paper, we propose a structure based image completion algorithm for object removal that produces visually plausible content with consistent structure and scene texture. We use an edge matching technique to infer the potential structure of the unknown region. Driven by the estimated structure, texture synthesis is performed automatically along the estimated curves. We evaluate the proposed method on different types of images: from highly structured indoor environment to the natural scenes. Our experimental results demonstrate satisfactory performance that can be potentially used for subsequent big data processing: 3D scene reconstruction and location recognition.