Lightweight and Progressively-Scalable Networks for Semantic Segmentation

TL;DR

This paper introduces LPS-Net, a lightweight, scalable network for semantic segmentation that progressively expands in complexity, achieving high accuracy and speed suitable for real-world deployment.

Contribution

The paper proposes a novel progressive scaling approach for lightweight networks, guided by three design principles, to optimize the speed-accuracy tradeoff in semantic segmentation.

Findings

LPS-Net outperforms several efficient methods on three datasets.

Achieves 73.4% mIoU on Cityscapes with 413.5FPS.

Improves performance and speed over STDC by 1.5% and 65%.

Abstract

Multi-scale learning frameworks have been regarded as a capable class of models to boost semantic segmentation. The problem nevertheless is not trivial especially for the real-world deployments, which often demand high efficiency in inference latency. In this paper, we thoroughly analyze the design of convolutional blocks (the type of convolutions and the number of channels in convolutions), and the ways of interactions across multiple scales, all from lightweight standpoint for semantic segmentation. With such in-depth comparisons, we conclude three principles, and accordingly devise Lightweight and Progressively-Scalable Networks (LPS-Net) that novelly expands the network complexity in a greedy manner. Technically, LPS-Net first capitalizes on the principles to build a tiny network. Then, LPS-Net progressively scales the tiny network to larger ones by expanding a single dimension (the number of convolutional blocks, the number of channels, or the input resolution) at one time to meet the best speed/accuracy tradeoff. Extensive experiments conducted on three datasets consistently demonstrate the superiority of LPS-Net over several efficient semantic segmentation methods. More remarkably, our LPS-Net achieves 73.4% mIoU on Cityscapes test set, with the speed of 413.5FPS on an NVIDIA GTX 1080Ti, leading to a performance improvement by 1.5% and a 65% speed-up against the state-of-the-art STDC. Code is available at \url{https://github.com/YihengZhang-CV/LPS-Net}.