ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time

TL;DR

ContextNet is a real-time semantic segmentation neural network that efficiently combines global context and detailed information, achieving high accuracy and speed suitable for embedded devices.

Contribution

It introduces a novel architecture that integrates low-resolution global context with high-resolution details, optimizing for real-time performance and low memory usage.

Findings

Achieves 66.1% accuracy on Cityscapes dataset.

Operates at 18.3 frames per second at full resolution.

Supports pipelined computations for streamed data.

Abstract

Modern deep learning architectures produce highly accurate results on many challenging semantic segmentation datasets. State-of-the-art methods are, however, not directly transferable to real-time applications or embedded devices, since naive adaptation of such systems to reduce computational cost (speed, memory and energy) causes a significant drop in accuracy. We propose ContextNet, a new deep neural network architecture which builds on factorized convolution, network compression and pyramid representation to produce competitive semantic segmentation in real-time with low memory requirement. ContextNet combines a deep network branch at low resolution that captures global context information efficiently with a shallow branch that focuses on high-resolution segmentation details. We analyse our network in a thorough ablation study and present results on the Cityscapes dataset, achieving 66.1% accuracy at 18.3 frames per second at full (1024x2048) resolution (41.9 fps with pipelined computations for streamed data).