SpaceMeshLab: Spatial Context Memoization and Meshgrid Atrous Convolution Consensus for Semantic Segmentation

TL;DR

SpaceMeshLab introduces a novel spatial context memoization method and a meshgrid-based atrous convolution scheme to improve multi-scale semantic segmentation, achieving state-of-the-art results on Cityscapes and Pascal-Context datasets.

Contribution

The paper proposes SpaM for spatial context retention and MetroCon^2 for fine-grained multi-scale context, addressing misalignment issues in semantic segmentation networks.

Findings

Achieves 82.0% mIoU on Cityscapes test set.

Achieves 53.5% mIoU on Pascal-Context validation set.

Introduces meshgrid-like scattered dilation rates for better multi-scale context.

Abstract

Semantic segmentation networks adopt transfer learning from image classification networks which occurs a shortage of spatial context information. For this reason, we propose Spatial Context Memoization (SpaM), a bypassing branch for spatial context by retaining the input dimension and constantly communicating its spatial context and rich semantic information mutually with the backbone network. Multi-scale context information for semantic segmentation is crucial for dealing with diverse sizes and shapes of target objects in the given scene. Conventional multi-scale context scheme adopts multiple effective receptive fields by multiple dilation rates or pooling operations, but often suffer from misalignment problem with respect to the target pixel. To this end, we propose Meshgrid Atrous Convolution Consensus (MetroCon^2) which brings multi-scale scheme into fine-grained multi-scale object context using convolutions with meshgrid-like scattered dilation rates. SpaceMeshLab (ResNet-101 + SpaM + MetroCon^2) achieves 82.0% mIoU in Cityscapes test and 53.5% mIoU on Pascal-Context validation set.