Asymmetric Non-local Neural Networks for Semantic Segmentation

TL;DR

This paper introduces Asymmetric Non-local Neural Networks with specialized blocks that significantly reduce computation and memory use while achieving state-of-the-art results in semantic segmentation.

Contribution

It proposes novel asymmetric pyramid and fusion non-local blocks that improve efficiency and performance in semantic segmentation tasks.

Findings

Achieved 81.3 mIoU on Cityscapes test set.

APNB is 6 times faster and 28 times more memory-efficient than traditional non-local blocks.

Demonstrated effectiveness and efficiency through extensive experiments.

Abstract

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: https://github.com/MendelXu/ANN.git.