NeuralArTS: Structuring Neural Architecture Search with Type Theory

TL;DR

NeuralArTS introduces a type-theoretic framework to structure neural architecture search spaces, aiming to improve the efficiency and organization of NAS by categorizing operations systematically.

Contribution

The paper proposes NeuralArTS, a novel type system for NAS that structures the search space, facilitating more efficient and organized architecture development.

Findings

NeuralArTS effectively categorizes network operations.

Application to convolutional layers demonstrates practical utility.

Provides future directions for NAS research.

Abstract

Neural Architecture Search (NAS) algorithms automate the task of finding optimal deep learning architectures given an initial search space of possible operations. Developing these search spaces is usually a manual affair with pre-optimized search spaces being more efficient, rather than searching from scratch. In this paper we present a new framework called Neural Architecture Type System (NeuralArTS) that categorizes the infinite set of network operations in a structured type system. We further demonstrate how NeuralArTS can be applied to convolutional layers and propose several future directions.