Differentiating Features for Scene Segmentation Based on Dedicated Attention Mechanisms

TL;DR

This paper introduces dedicated attention mechanisms to differentiate features for scene segmentation, improving accuracy while reducing complexity, demonstrated on Cityscapes and PASCAL VOC 2012 datasets.

Contribution

Proposes two novel attention modules to optimize high-level and low-level features in scene segmentation, with a simplified architecture that achieves high accuracy without pre-training.

Findings

Achieves 82.3% Mean IoU on PASCAL VOC 2012 without pre-training.

Reduces model complexity by avoiding redundant modules.

Effective differentiation of features enhances segmentation performance.

Abstract

Semantic segmentation is a challenge in scene parsing. It requires both context information and rich spatial information. In this paper, we differentiate features for scene segmentation based on dedicated attention mechanisms (DF-DAM), and two attention modules are proposed to optimize the high-level and low-level features in the encoder, respectively. Specifically, we use the high-level and low-level features of ResNet as the source of context information and spatial information, respectively, and optimize them with attention fusion module and 2D position attention module, respectively. For attention fusion module, we adopt dual channel weight to selectively adjust the channel map for the highest two stage features of ResNet, and fuse them to get context information. For 2D position attention module, we use the context information obtained by attention fusion module to assist the selection of the lowest-stage features of ResNet as supplementary spatial information. Finally, the two sets of information obtained by the two modules are simply fused to obtain the prediction. We evaluate our approach on Cityscapes and PASCAL VOC 2012 datasets. In particular, there aren't complicated and redundant processing modules in our architecture, which greatly reduces the complexity, and we achieving 82.3% Mean IoU on PASCAL VOC 2012 test dataset without pre-training on MS-COCO dataset.