Window Detection In Facade Imagery: A Deep Learning Approach Using Mask R-CNN

TL;DR

This paper presents a deep learning method using Mask R-CNN for detecting and segmenting windows in facade images, achieving state-of-the-art results with limited training data.

Contribution

The study demonstrates effective window detection in facade imagery using transfer learning with Mask R-CNN, without requiring extensive datasets or post-processing.

Findings

Achieved competitive accuracy with small dataset

Utilized transfer learning and augmentation effectively

No post-optimization needed for high-quality segmentation

Abstract

The parsing of windows in building facades is a long-desired but challenging task in computer vision. It is crucial to urban analysis, semantic reconstruction, lifecycle analysis, digital twins, and scene parsing amongst other building-related tasks that require high-quality semantic data. This article investigates the usage of the mask R-CNN framework to be used for window detection of facade imagery input. We utilize transfer learning to train our proposed method on COCO weights with our own collected dataset of street view images of facades to produce instance segmentations of our new window class. Experimental results show that our suggested approach with a relatively small dataset trains the network only with transfer learning and augmentation achieves results on par with prior state-of-the-art window detection approaches, even without post-optimization techniques.