Progressive Semantic Segmentation

TL;DR

MagNet is a multi-scale framework for high-resolution image segmentation that progressively refines segmentation maps through multiple magnification levels, overcoming memory constraints and preserving fine details.

Contribution

This work introduces MagNet, a novel multi-stage, multi-scale approach that improves high-resolution image segmentation by progressively refining details at increasing magnification levels.

Findings

MagNet outperforms state-of-the-art methods on urban, aerial, and medical datasets.

The multi-stage approach effectively balances memory usage and detail preservation.

Progressive refinement leads to more accurate segmentation results.

Abstract

The objective of this work is to segment high-resolution images without overloading GPU memory usage or losing the fine details in the output segmentation map. The memory constraint means that we must either downsample the big image or divide the image into local patches for separate processing. However, the former approach would lose the fine details, while the latter can be ambiguous due to the lack of a global picture. In this work, we present MagNet, a multi-scale framework that resolves local ambiguity by looking at the image at multiple magnification levels. MagNet has multiple processing stages, where each stage corresponds to a magnification level, and the output of one stage is fed into the next stage for coarse-to-fine information propagation. Each stage analyzes the image at a higher resolution than the previous stage, recovering the previously lost details due to the lossy downsampling step, and the segmentation output is progressively refined through the processing stages. Experiments on three high-resolution datasets of urban views, aerial scenes, and medical images show that MagNet consistently outperforms the state-of-the-art methods by a significant margin.