Building Facade Parsing R-CNN

TL;DR

This paper introduces Facade R-CNN, a novel model for pixel-level building facade parsing from deformed, street-view images, outperforming existing models and providing a new annotated dataset for the task.

Contribution

The paper presents Facade R-CNN with a transconv module, generalized bounding box detection, and convex regularization, along with a new Oxford RobotCar Facade dataset for street-view facade parsing.

Findings

Facade R-CNN outperforms state-of-the-art models.

New Oxford RobotCar Facade dataset with 500 annotated images.

Enhanced parsing accuracy on deformed street-view facades.

Abstract

Building facade parsing, which predicts pixel-level labels for building facades, has applications in computer vision perception for autonomous vehicle (AV) driving. However, instead of a frontal view, an on-board camera of an AV captures a deformed view of the facade of the buildings on both sides of the road the AV is travelling on, due to the camera perspective. We propose Facade R-CNN, which includes a transconv module, generalized bounding box detection, and convex regularization, to perform parsing of deformed facade views. Experiments demonstrate that Facade R-CNN achieves better performance than the current state-of-the-art facade parsing models, which are primarily developed for frontal views. We also publish a new building facade parsing dataset derived from the Oxford RobotCar dataset, which we call the Oxford RobotCar Facade dataset. This dataset contains 500 street-view images from the Oxford RobotCar dataset augmented with accurate annotations of building facade objects. The published dataset is available at https://github.com/sijieaaa/Oxford-RobotCar-Facade