Real-Time Semantic Segmentation via Auto Depth, Downsampling Joint Decision and Feature Aggregation

TL;DR

This paper introduces AutoRTNet, a neural architecture search framework that automatically designs network depth, downsampling, and feature aggregation for real-time semantic segmentation, achieving high accuracy and speed.

Contribution

AutoRTNet is the first framework to jointly search for depth, downsampling, and feature aggregation strategies in real-time segmentation networks.

Findings

Achieves 73.9% mIoU on Cityscapes test set

Runs at 110 FPS on NVIDIA TitanXP

Automatically designs efficient network architectures

Abstract

To satisfy the stringent requirements on computational resources in the field of real-time semantic segmentation, most approaches focus on the hand-crafted design of light-weight segmentation networks. Recently, Neural Architecture Search (NAS) has been used to search for the optimal building blocks of networks automatically, but the network depth, downsampling strategy, and feature aggregation way are still set in advance by trial and error. In this paper, we propose a joint search framework, called AutoRTNet, to automate the design of these strategies. Specifically, we propose hyper-cells to jointly decide the network depth and downsampling strategy, and an aggregation cell to achieve automatic multi-scale feature aggregation. Experimental results show that AutoRTNet achieves 73.9% mIoU on the Cityscapes test set and 110.0 FPS on an NVIDIA TitanXP GPU card with 768x1536 input images.