# Prune and Replace NAS

## TL;DR

PR-DARTS introduces a NAS method that efficiently searches a much larger operation space by pruning and replacing candidates, achieving competitive results within a single day.

## Contribution

It presents a novel NAS approach that explores a significantly larger search space through progressive pruning and replacement, improving search efficiency and performance.

## Key findings

- Achieves 2.51% test error on CIFAR-10
- Achieves 15.53% test error on CIFAR-100
- Search space is 150 times larger than DARTS baseline

## Abstract

While recent NAS algorithms are thousands of times faster than the pioneering works, it is often overlooked that they use fewer candidate operations, resulting in a significantly smaller search space. We present PR-DARTS, a NAS algorithm that discovers strong network configurations in a much larger search space and a single day. A small candidate operation pool is used, from which candidates are progressively pruned and replaced with better performing ones. Experiments on CIFAR-10 and CIFAR-100 achieve 2.51% and 15.53% test error, respectively, despite searching in a space where each cell has 150 times as many possible configurations than in the DARTS baseline. Code is available at https://github.com/cogsys-tuebingen/prdarts