Conceptual Expansion Neural Architecture Search (CENAS)

TL;DR

CENAS introduces a transfer learning-based neural architecture search method that efficiently finds better models faster by adding features rather than just modifying existing ones, outperforming standard methods.

Contribution

It combines transfer learning with neural architecture search to enable faster, more efficient model discovery with feature addition capabilities.

Findings

Outperforms standard NAS and transfer learning in efficiency.

Achieves better performance with fewer parameters.

Demonstrates effectiveness across various transfer tasks.

Abstract

Architecture search optimizes the structure of a neural network for some task instead of relying on manual authoring. However, it is slow, as each potential architecture is typically trained from scratch. In this paper we present an approach called Conceptual Expansion Neural Architecture Search (CENAS) that combines a sample-efficient, computational creativity-inspired transfer learning approach with neural architecture search. This approach finds models faster than naive architecture search via transferring existing weights to approximate the parameters of the new model. It outperforms standard transfer learning by allowing for the addition of features instead of only modifying existing features. We demonstrate that our approach outperforms standard neural architecture search and transfer learning methods in terms of efficiency, performance, and parameter counts on a variety of transfer learning tasks.