Best Practices for Scientific Research on Neural Architecture Search

TL;DR

This paper discusses the challenges in evaluating neural architecture search methods and proposes best practices to improve the quality and reliability of empirical research in NAS.

Contribution

It introduces a comprehensive checklist of best practices for conducting and reporting NAS experiments to enhance scientific rigor.

Findings

Identifies common issues in NAS empirical evaluations

Provides guidelines to improve reproducibility and comparability

Offers a checklist for NAS research best practices

Abstract

Finding a well-performing architecture is often tedious for both DL practitioners and researchers, leading to tremendous interest in the automation of this task by means of neural architecture search (NAS). Although the community has made major strides in developing better NAS methods, the quality of scientific empirical evaluations in the young field of NAS is still lacking behind that of other areas of machine learning. To address this issue, we describe a set of possible issues and ways to avoid them, leading to the NAS best practices checklist available at http://automl.org/nas_checklist.pdf.