Denoised Non-Local Neural Network for Semantic Segmentation

TL;DR

This paper introduces a denoising approach for non-local neural networks in semantic segmentation, improving attention map reliability by reducing noise and enhancing accuracy without external data.

Contribution

The paper proposes a novel Denoised Non-Local Network with Global Rectifying and Local Retention modules to effectively eliminate attention noise in semantic segmentation.

Findings

Achieves state-of-the-art 83.5% mIoU on Cityscapes.

Attains 46.69% mIoU on ADE20K.

Improves segmentation accuracy by denoising attention maps.

Abstract

The non-local network has become a widely used technique for semantic segmentation, which computes an attention map to measure the relationships of each pixel pair. However, most of the current popular non-local models tend to ignore the phenomenon that the calculated attention map appears to be very noisy, containing inter-class and intra-class inconsistencies, which lowers the accuracy and reliability of the non-local methods. In this paper, we figuratively denote these inconsistencies as attention noises and explore the solutions to denoise them. Specifically, we inventively propose a Denoised Non-Local Network (Denoised NL), which consists of two primary modules, i.e., the Global Rectifying (GR) block and the Local Retention (LR) block, to eliminate the inter-class and intra-class noises respectively. First, GR adopts the class-level predictions to capture a binary map to distinguish whether the selected two pixels belong to the same category. Second, LR captures the ignored local dependencies and further uses them to rectify the unwanted hollows in the attention map. The experimental results on two challenging semantic segmentation datasets demonstrate the superior performance of our model. Without any external training data, our proposed Denoised NL can achieve the state-of-the-art performance of 83.5\% and 46.69\% mIoU on Cityscapes and ADE20K, respectively.