Raster2Seq: Polygon Sequence Generation for Floorplan Reconstruction

TL;DR

Raster2Seq is a novel sequence-to-sequence approach for reconstructing detailed, structured floorplan vector graphics from raster images, effectively handling complex indoor layouts with many rooms and diverse geometries.

Contribution

It introduces an autoregressive decoder with learnable anchors for flexible, accurate polygon sequence generation from floorplan images, advancing state-of-the-art performance.

Findings

Achieves state-of-the-art results on Structure3D, CubiCasa5K, and Raster2Graph datasets.

Demonstrates strong generalization to diverse and complex datasets like WAFFLE.

Abstract

Reconstructing a structured vector-graphics representation from a rasterized floorplan image is typically an important prerequisite for computational tasks involving floorplans such as automated understanding or CAD workflows. However, existing techniques struggle in faithfully generating the structure and semantics conveyed by complex floorplans that depict large indoor spaces with many rooms and a varying numbers of polygon corners. To this end, we propose Raster2Seq, framing floorplan reconstruction as a sequence-to-sequence task in which floorplan elements--such as rooms, windows, and doors--are represented as labeled polygon sequences that jointly encode geometry and semantics. Our approach introduces an autoregressive decoder that learns to predict the next corner conditioned on image features and previously generated corners using guidance from learnable anchors. These anchors represent spatial coordinates in image space, hence allowing for effectively directing the attention mechanism to focus on informative image regions. By embracing the autoregressive mechanism, our method offers flexibility in the output format, enabling for efficiently handling complex floorplans with numerous rooms and diverse polygon structures. Our method achieves state-of-the-art performance on standard benchmarks such as Structure3D, CubiCasa5K, and Raster2Graph, while also demonstrating strong generalization to more challenging datasets like WAFFLE, which contain diverse room structures and complex geometric variations.