ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

TL;DR

ESPNet is a novel, highly efficient neural network for semantic segmentation that significantly reduces computation and memory requirements while maintaining competitive accuracy, enabling real-time processing on resource-constrained devices.

Contribution

The paper introduces ESPNet, a new convolutional module and network architecture that achieves high efficiency and speed for semantic segmentation, outperforming existing efficient CNNs under resource constraints.

Findings

ESPNet is 22 times faster than PSPNet on GPU.

ESPNet is 180 times smaller than PSPNet.

ESPNet outperforms MobileNet, ShuffleNet, and ENet in efficiency and accuracy.

Abstract

We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints. ESPNet is based on a new convolutional module, efficient spatial pyramid (ESP), which is efficient in terms of computation, memory, and power. ESPNet is 22 times faster (on a standard GPU) and 180 times smaller than the state-of-the-art semantic segmentation network PSPNet, while its category-wise accuracy is only 8% less. We evaluated ESPNet on a variety of semantic segmentation datasets including Cityscapes, PASCAL VOC, and a breast biopsy whole slide image dataset. Under the same constraints on memory and computation, ESPNet outperforms all the current efficient CNN networks such as MobileNet, ShuffleNet, and ENet on both standard metrics and our newly introduced performance metrics that measure efficiency on edge devices. Our network can process high resolution images at a rate of 112 and 9 frames per second on a standard GPU and edge device, respectively.