Tiling and Stitching Segmentation Output for Remote Sensing: Basic Challenges and Recommendations

TL;DR

This paper proposes increasing CNN input size during inference to improve segmentation of large remote sensing images, reducing stitching issues and inference time, with demonstrated accuracy gains and winning a competition.

Contribution

The paper introduces a simple method of enlarging CNN input during inference to mitigate stitching artifacts and improve efficiency in remote sensing image segmentation.

Findings

Reduces label inference time significantly.

Yields modest accuracy improvements.

Contributed to winning the INRIA building labeling competition.

Abstract

In this work we consider the application of convolutional neural networks (CNNs) for pixel-wise labeling (a.k.a., semantic segmentation) of remote sensing imagery (e.g., aerial color or hyperspectral imagery). Remote sensing imagery is usually stored in the form of very large images, referred to as "tiles", which are too large to be segmented directly using most CNNs and their associated hardware. As a result, during label inference, smaller sub-images, called "patches", are processed individually and then "stitched" (concatenated) back together to create a tile-sized label map. This approach suffers from computational ineffiency and can result in discontinuities at output boundaries. We propose a simple alternative approach in which the input size of the CNN is dramatically increased only during label inference. This does not avoid stitching altogether, but substantially mitigates its limitations. We evaluate the performance of the proposed approach against a vonventional stitching approach using two popular segmentation CNN models and two large-scale remote sensing imagery datasets. The results suggest that the proposed approach substantially reduces label inference time, while also yielding modest overall label accuracy increases. This approach contributed to our wining entry (overall performance) in the INRIA building labeling competition.