Guided Upsampling Network for Real-Time Semantic Segmentation

TL;DR

This paper introduces the Guided Upsampling Network, a multiresolution architecture with a learnable upsampling module that enhances real-time semantic segmentation performance on high-resolution images.

Contribution

The paper proposes a novel Guided Upsampling Module (GUM) that can be integrated into existing architectures to improve upsampling with minimal computational overhead.

Findings

Achieves state-of-the-art performance on Cityscapes dataset.

Operates in real-time on high-resolution images.

Enhances upsampling quality with learnable transformations.

Abstract

Semantic segmentation architectures are mainly built upon an encoder-decoder structure. These models perform subsequent downsampling operations in the encoder. Since operations on high-resolution activation maps are computationally expensive, usually the decoder produces output segmentation maps by upsampling with parameters-free operators like bilinear or nearest-neighbor. We propose a Neural Network named Guided Upsampling Network which consists of a multiresolution architecture that jointly exploits high-resolution and large context information. Then we introduce a new module named Guided Upsampling Module (GUM) that enriches upsampling operators by introducing a learnable transformation for semantic maps. It can be plugged into any existing encoder-decoder architecture with little modifications and low additional computation cost. We show with quantitative and qualitative experiments how our network benefits from the use of GUM module. A comprehensive set of experiments on the publicly available Cityscapes dataset demonstrates that Guided Upsampling Network can efficiently process high-resolution images in real-time while attaining state-of-the art performances.