Unified Functional Hashing in Automatic Machine Learning

TL;DR

This paper introduces a unified functional hashing technique that significantly accelerates AutoML search processes by efficiently detecting equivalent candidates and avoiding redundant evaluations, applicable across various representations.

Contribution

The paper presents a novel unified functional hash and caching method that improves AutoML efficiency by identifying equivalent solutions across different representations.

Findings

Dramatic speedups in AutoML domains like neural architecture search.

Effective detection of equivalent candidates reduces re-evaluation costs.

Analysis of hash collisions and noise impacts on search accuracy.

Abstract

The field of Automatic Machine Learning (AutoML) has recently attained impressive results, including the discovery of state-of-the-art machine learning solutions, such as neural image classifiers. This is often done by applying an evolutionary search method, which samples multiple candidate solutions from a large space and evaluates the quality of each candidate through a long training process. As a result, the search tends to be slow. In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present. The central idea is to detect by hashing when the search method produces equivalent candidates, which occurs very frequently, and this way avoid their costly re-evaluation. Our hash is "functional" in that it identifies equivalent candidates even if they were represented or coded differently, and it is "unified" in that the same algorithm can hash arbitrary representations; e.g. compute graphs, imperative code, or lambda functions. As evidence, we show dramatic improvements on multiple AutoML domains, including neural architecture search and algorithm discovery. Finally, we consider the effect of hash collisions, evaluation noise, and search distribution through empirical analysis. Altogether, we hope this paper may serve as a guide to hashing techniques in AutoML.