# Template-Based Automatic Search of Compact Semantic Segmentation Architectures

## TL;DR

This paper introduces a novel template-based neural architecture search method for dense semantic segmentation that efficiently finds lightweight, high-performing models starting from pre-trained classification networks, using reinforcement learning.

## Contribution

It proposes a new approach that builds segmentation architectures from templates and predicts structure and operations with a recurrent network, enabling efficient search for compact models.

## Key findings

- Achieved 63.2% mean IoU on CamVid with 270K parameters.
- Achieved 67.8% mean IoU on CityScapes with 270K parameters.
- The method outperforms existing automatic search approaches for dense segmentation.

## Abstract

Automatic search of neural architectures for various vision and natural language tasks is becoming a prominent tool as it allows to discover high-performing structures on any dataset of interest. Nevertheless, on more difficult domains, such as dense per-pixel classification, current automatic approaches are limited in their scope - due to their strong reliance on existing image classifiers they tend to search only for a handful of additional layers with discovered architectures still containing a large number of parameters. In contrast, in this work we propose a novel solution able to find light-weight and accurate segmentation architectures starting from only few blocks of a pre-trained classification network. To this end, we progressively build up a methodology that relies on templates of sets of operations, predicts which template and how many times should be applied at each step, while also generating the connectivity structure and downsampling factors. All these decisions are being made by a recurrent neural network that is rewarded based on the score of the emitted architecture on the holdout set and trained using reinforcement learning. One discovered architecture achieves 63.2% mean IoU on CamVid and 67.8% on CityScapes having only 270K parameters. Pre-trained models and the search code are available at https://github.com/DrSleep/nas-segm-pytorch.