SegForestNet: Spatial-Partitioning-Based Aerial Image Segmentation

TL;DR

SegForestNet introduces a novel deep learning model that predicts polygonal regions for aerial image segmentation using binary space partitioning trees, improving accuracy for small objects and leveraging spatial partitioning.

Contribution

The paper presents a new BSP tree-based segmentation model with a differentiable renderer, a specialized loss function, and multi-tree prediction capability, advancing aerial image segmentation techniques.

Findings

Better predictions for small rectangular objects like cars.

Model advantages diminish under optimal training conditions.

Enhanced spatial partitioning improves segmentation accuracy.

Abstract

Aerial image segmentation is the basis for applications such as automatically creating maps or tracking deforestation. In true orthophotos, which are often used in these applications, many objects and regions can be approximated well by polygons. However, this fact is rarely exploited by state-of-the-art semantic segmentation models. Instead, most models allow unnecessary degrees of freedom in their predictions by allowing arbitrary region shapes. We therefore present a refinement of our deep learning model which predicts binary space partitioning trees, an efficient polygon representation. The refinements include a new feature decoder architecture and a new differentiable BSP tree renderer which both avoid vanishing gradients. Additionally, we designed a novel loss function specifically designed to improve the spatial partitioning defined by the predicted trees. Furthermore, our expanded model can predict multiple trees at once and thus can predict class-specific segmentations. As an additional contribution, we investigate the impact of a non-optimal training process in comparison to an optimized training process. While model architectures optimized for aerial images, such as PFNet or our own model, show an advantage under non-optimal conditions, this advantage disappears under optimal training conditions. Despite this observation, our model still makes better predictions for small rectangular objects, e.g., cars.