RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings

TL;DR

RoIPoly is a novel vectorized building outline extraction method that uses vertex queries and logit embeddings to improve efficiency and reduce redundancy, outperforming existing approaches on multiple datasets.

Contribution

The paper introduces RoIPoly, a region-of-interest query-based approach with learnable logit embeddings for efficient, redundant-free polygon extraction from aerial imagery.

Findings

Outperforms existing methods on the CrowdAI dataset, especially for small buildings.

Achieves second-best performance on 2D floorplan reconstruction in Structured3D.

Reduces computational overhead and eliminates post-processing for redundancy.

Abstract

Polygonal building outlines are crucial for geographic and cartographic applications. The existing approaches for outline extraction from aerial or satellite imagery are typically decomposed into subtasks, e.g., building masking and vectorization, or treat this task as a sequence-to-sequence prediction of ordered vertices. The former lacks efficiency, and the latter often generates redundant vertices, both resulting in suboptimal performance. To handle these issues, we propose a novel Region-of-Interest (RoI) query-based approach called RoIPoly. Specifically, we formulate each vertex as a query and constrain the query attention on the most relevant regions of a potential building, yielding reduced computational overhead and more efficient vertex level interaction. Moreover, we introduce a novel learnable logit embedding to facilitate vertex classification on the attention map; thus, no post-processing is needed for redundant vertex removal. We evaluated our method on the vectorized building outline extraction dataset CrowdAI and the 2D floorplan reconstruction dataset Structured3D. On the CrowdAI dataset, RoIPoly with a ResNet50 backbone outperforms existing methods with the same or better backbones on most MS-COCO metrics, especially on small buildings, and achieves competitive results in polygon quality and vertex redundancy without any post-processing. On the Structured3D dataset, our method achieves the second-best performance on most metrics among existing methods dedicated to 2D floorplan reconstruction, demonstrating our cross-domain generalization capability. The code will be released upon acceptance of this paper.