Global Aggregation then Local Distribution for Scene Parsing

TL;DR

This paper introduces GALD, a novel module that combines global aggregation with local distribution to improve semantic segmentation accuracy, especially around boundaries and small objects.

Contribution

The paper proposes a new local distribution module that enhances global aggregation methods by adaptively modeling local relationships, leading to state-of-the-art results in semantic segmentation.

Findings

Achieved new state-of-the-art performance on Cityscapes, ADE20K, Pascal Context, Camvid, and COCO-stuff.

The approach is modular, end-to-end trainable, and easily integrated into existing networks.

Code and models are publicly available for further research.

Abstract

Modelling long-range contextual relationships is critical for pixel-wise prediction tasks such as semantic segmentation. However, convolutional neural networks (CNNs) are inherently limited to model such dependencies due to the naive structure in its building modules (\eg, local convolution kernel). While recent global aggregation methods are beneficial for long-range structure information modelling, they would oversmooth and bring noise to the regions containing fine details (\eg,~boundaries and small objects), which are very much cared for the semantic segmentation task. To alleviate this problem, we propose to explore the local context for making the aggregated long-range relationship being distributed more accurately in local regions. In particular, we design a novel local distribution module which models the affinity map between global and local relationship for each pixel adaptively. Integrating existing global aggregation modules, we show that our approach can be modularized as an end-to-end trainable block and easily plugged into existing semantic segmentation networks, giving rise to the \emph{GALD} networks. Despite its simplicity and versatility, our approach allows us to build new state of the art on major semantic segmentation benchmarks including Cityscapes, ADE20K, Pascal Context, Camvid and COCO-stuff. Code and trained models are released at \url{https://github.com/lxtGH/GALD-DGCNet} to foster further research.