HiT: Building Mapping with Hierarchical Transformers

TL;DR

HiT introduces a hierarchical transformer-based approach for building mapping from high-resolution remote sensing images, producing accurate vector polygons and bounding boxes in an end-to-end manner, outperforming existing methods.

Contribution

The paper presents a novel end-to-end building mapping method using Hierarchical Transformers with a polygon head that predicts serialized vertices, improving accuracy and simplicity over traditional multi-step approaches.

Findings

Achieves state-of-the-art results on CrowdAI and Inria datasets.

Outperforms existing methods in instance segmentation and polygonal metrics.

Qualitative results show robustness in complex scenes.

Abstract

Deep learning-based methods have been extensively explored for automatic building mapping from high-resolution remote sensing images over recent years. While most building mapping models produce vector polygons of buildings for geographic and mapping systems, dominant methods typically decompose polygonal building extraction in some sub-problems, including segmentation, polygonization, and regularization, leading to complex inference procedures, low accuracy, and poor generalization. In this paper, we propose a simple and novel building mapping method with Hierarchical Transformers, called HiT, improving polygonal building mapping quality from high-resolution remote sensing images. HiT builds on a two-stage detection architecture by adding a polygon head parallel to classification and bounding box regression heads. HiT simultaneously outputs building bounding boxes and vector polygons, which is fully end-to-end trainable. The polygon head formulates a building polygon as serialized vertices with the bidirectional characteristic, a simple and elegant polygon representation avoiding the start or end vertex hypothesis. Under this new perspective, the polygon head adopts a transformer encoder-decoder architecture to predict serialized vertices supervised by the designed bidirectional polygon loss. Furthermore, a hierarchical attention mechanism combined with convolution operation is introduced in the encoder of the polygon head, providing more geometric structures of building polygons at vertex and edge levels. Comprehensive experiments on two benchmarks (the CrowdAI and Inria datasets) demonstrate that our method achieves a new state-of-the-art in terms of instance segmentation and polygonal metrics compared with state-of-the-art methods. Moreover, qualitative results verify the superiority and effectiveness of our model under complex scenes.