Scene restoration from scaffold occlusion using deep learning-based methods

TL;DR

This paper introduces a two-step deep learning approach combining segmentation and inpainting to restore construction scenes occluded by scaffolds, using a novel low-cost data synthesis method to overcome labeled data scarcity.

Contribution

It presents a new two-step method for scene restoration from scaffold occlusion and a low-cost data synthesis technique based on unlabeled data.

Findings

92% MIoU for scaffold segmentation

Over 82% SSIM for scene restoration

Effective in synthetic test scenarios

Abstract

The occlusion issues of computer vision (CV) applications in construction have attracted significant attention, especially those caused by the wide-coverage, crisscrossed, and immovable scaffold. Intuitively, removing the scaffold and restoring the occluded visual information can provide CV agents with clearer site views and thus help them better understand the construction scenes. Therefore, this study proposes a novel two-step method combining pixel-level segmentation and image inpainting for restoring construction scenes from scaffold occlusion. A low-cost data synthesis method based only on unlabeled data is developed to address the shortage dilemma of labeled data. Experiments on the synthesized test data show that the proposed method achieves performances of 92% mean intersection over union (MIoU) for scaffold segmentation and over 82% structural similarity (SSIM) for scene restoration from scaffold occlusion.