AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation

TL;DR

This paper introduces AF$_2$, an adaptive framework for aerial imagery segmentation that dynamically selects multi-scale features using a learnable module, significantly improving accuracy on standard benchmarks.

Contribution

The paper proposes a novel Adaptive Focus Framework with a learnable Adaptive Confidence Mechanism for better multi-scale feature utilization in aerial image segmentation.

Findings

Significant accuracy improvements on three aerial benchmarks.

Efficient segmentation comparable in speed to mainstream methods.

Effective adaptive selection of multi-scale features for diverse object sizes.

Abstract

As a specific semantic segmentation task, aerial imagery segmentation has been widely employed in high spatial resolution (HSR) remote sensing images understanding. Besides common issues (e.g. large scale variation) faced by general semantic segmentation tasks, aerial imagery segmentation has some unique challenges, the most critical one among which lies in foreground-background imbalance. There have been some recent efforts that attempt to address this issue by proposing sophisticated neural network architectures, since they can be used to extract informative multi-scale feature representations and increase the discrimination of object boundaries. Nevertheless, many of them merely utilize those multi-scale representations in ad-hoc measures but disregard the fact that the semantic meaning of objects with various sizes could be better identified via receptive fields of diverse ranges. In this paper, we propose Adaptive Focus Framework (AF$_2$), which adopts a hierarchical segmentation procedure and focuses on adaptively utilizing multi-scale representations generated by widely adopted neural network architectures. Particularly, a learnable module, called Adaptive Confidence Mechanism (ACM), is proposed to determine which scale of representation should be used for the segmentation of different objects. Comprehensive experiments show that AF$_2$ has significantly improved the accuracy on three widely used aerial benchmarks, as fast as the mainstream method.