Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone

TL;DR

This paper introduces a novel framework for building roof instance segmentation using domain-adapted pretraining, composite dual-backbones, and advanced data augmentation, achieving top performance in a major contest.

Contribution

It presents a new segmentation approach that combines domain adaptation, dual-backbones, and multimodal data fusion, improving robustness and accuracy over existing methods.

Findings

Achieved 50.6% mAP in IEEE DFC 2023 contest

Enhanced feature learning via domain-adapted pretraining and dual-backbones

Demonstrated the effectiveness of multimodal data fusion with optical and SAR imagery

Abstract

The diversity of building architecture styles of global cities situated on various landforms, the degraded optical imagery affected by clouds and shadows, and the significant inter-class imbalance of roof types pose challenges for designing a robust and accurate building roof instance segmentor. To address these issues, we propose an effective framework to fulfill semantic interpretation of individual buildings with high-resolution optical satellite imagery. Specifically, the leveraged domain adapted pretraining strategy and composite dual-backbone greatly facilitates the discriminative feature learning. Moreover, new data augmentation pipeline, stochastic weight averaging (SWA) training and instance segmentation based model ensemble in testing are utilized to acquire additional performance boost. Experiment results show that our approach ranks in the first place of the 2023 IEEE GRSS Data Fusion Contest (DFC) Track 1 test phase ($mAP_{50}$:50.6\%). Note-worthily, we have also explored the potential of multimodal data fusion with both optical satellite imagery and SAR data.