Real-time Semantic Image Segmentation via Spatial Sparsity

TL;DR

This paper introduces a real-time semantic image segmentation method that significantly reduces computational costs by exploiting spatial sparsity, enabling high-speed processing with minimal quality loss.

Contribution

The approach employs a two-column network with spatial sparsity to achieve 25x faster segmentation while maintaining competitive accuracy.

Findings

Processes 15 high-resolution images per second

Achieves 72.9% mean intersection-over-union on Cityscapes

Reduces computational costs by a factor of 25

Abstract

We propose an approach to semantic (image) segmentation that reduces the computational costs by a factor of 25 with limited impact on the quality of results. Semantic segmentation has a number of practical applications, and for most such applications the computational costs are critical. The method follows a typical two-column network structure, where one column accepts an input image, while the other accepts a half-resolution version of that image. By identifying specific regions in the full-resolution image that can be safely ignored, as well as carefully tailoring the network structure, we can process approximately 15 highresolution Cityscapes images (1024x2048) per second using a single GTX 980 video card, while achieving a mean intersection-over-union score of 72.9% on the Cityscapes test set.