Hierarchical Instance Mixing across Domains in Aerial Segmentation

TL;DR

This paper introduces Hierarchical Instance Mixing (HIMix), a novel domain adaptation method for aerial segmentation that addresses semantic disparity and structural variability, achieving state-of-the-art results on LoveDA.

Contribution

The paper proposes HIMix, a new mixing strategy and twin-head architecture, specifically designed for aerial domain adaptation, improving over existing methods.

Findings

HIMix outperforms current state-of-the-art on LoveDA benchmark.

The hierarchical mixing better handles semantic imbalance in aerial scenes.

Twin-head architecture enhances segmentation accuracy.

Abstract

We investigate the task of unsupervised domain adaptation in aerial semantic segmentation and discover that the current state-of-the-art algorithms designed for autonomous driving based on domain mixing do not translate well to the aerial setting. This is due to two factors: (i) a large disparity in the extension of the semantic categories, which causes a domain imbalance in the mixed image, and (ii) a weaker structural consistency in aerial scenes than in driving scenes since the same scene might be viewed from different perspectives and there is no well-defined and repeatable structure of the semantic elements in the images. Our solution to these problems is composed of: (i) a new mixing strategy for aerial segmentation across domains called Hierarchical Instance Mixing (HIMix), which extracts a set of connected components from each semantic mask and mixes them according to a semantic hierarchy and, (ii) a twin-head architecture in which two separate segmentation heads are fed with variations of the same images in a contrastive fashion to produce finer segmentation maps. We conduct extensive experiments on the LoveDA benchmark, where our solution outperforms the current state-of-the-art.