Cross-Layer Attentive Feature Upsampling for Low-latency Semantic Segmentation

TL;DR

This paper introduces Guided Attentive Interpolation (GAI), a novel method for high-resolution feature upsampling in semantic segmentation that improves accuracy and efficiency, enabling low-latency real-time applications.

Contribution

The paper proposes GAI, a new adaptive interpolation technique that models spatial and semantic relations for better high-resolution feature generation in segmentation networks.

Findings

Achieves state-of-the-art low-latency segmentation accuracy

Runs at 22.3 FPS on Cityscapes with high mIoU

Outperforms existing methods in efficiency and accuracy

Abstract

Semantic segmentation is a fundamental problem in computer vision and it requires high-resolution feature maps for dense prediction. Current coordinate-guided low-resolution feature interpolation methods, e.g., bilinear interpolation, produce coarse high-resolution features which suffer from feature misalignment and insufficient context information. Moreover, enriching semantics to high-resolution features requires a high computation burden, so that it is challenging to meet the requirement of lowlatency inference. We propose a novel Guided Attentive Interpolation (GAI) method to adaptively interpolate fine-grained high-resolution features with semantic features to tackle these issues. Guided Attentive Interpolation determines both spatial and semantic relations of pixels from features of different resolutions and then leverages these relations to interpolate high-resolution features with rich semantics. GAI can be integrated with any deep convolutional network for efficient semantic segmentation. In experiments, the GAI-based semantic segmentation networks, i.e., GAIN, can achieve78.8 mIoU with 22.3 FPS on Cityscapes and 80.6 mIoU with 64.5 on CamVid using an NVIDIA 1080Ti GPU, which are the new state-of-the-art results of low-latency semantic segmentation. Code and models are available at: https://github.com/hustvl/simpleseg.