ResBuilder: Automated Learning of Depth with Residual Structures

TL;DR

ResBuilder is an automated neural architecture search method that designs ResNet architectures from scratch or modifies existing ones, achieving high accuracy with reduced computational cost across multiple datasets.

Contribution

It introduces a novel NAS algorithm for ResNet architectures that generalizes well across datasets and industrial applications, with minimal dataset-specific tuning.

Findings

Achieves near state-of-the-art accuracy on image classification datasets.

Reduces computational cost compared to standard ResNets.

Generalizes to industrial datasets with minimal tuning.

Abstract

In this work, we develop a neural architecture search algorithm, termed Resbuilder, that develops ResNet architectures from scratch that achieve high accuracy at moderate computational cost. It can also be used to modify existing architectures and has the capability to remove and insert ResNet blocks, in this way searching for suitable architectures in the space of ResNet architectures. In our experiments on different image classification datasets, Resbuilder achieves close to state-of-the-art performance while saving computational cost compared to off-the-shelf ResNets. Noteworthy, we once tune the parameters on CIFAR10 which yields a suitable default choice for all other datasets. We demonstrate that this property generalizes even to industrial applications by applying our method with default parameters on a proprietary fraud detection dataset.