Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization

TL;DR

This paper introduces a latency-aware neural architecture search method using multi-objective Bayesian optimization to efficiently balance model accuracy and on-device latency for large-scale natural language models.

Contribution

It applies advanced Bayesian optimization techniques to optimize neural architectures considering multiple objectives, specifically latency and accuracy, in a production environment.

Findings

Effective trade-off exploration between latency and accuracy.

Improved neural architecture search efficiency.

Demonstrated results on Facebook's natural language models.

Abstract

When tuning the architecture and hyperparameters of large machine learning models for on-device deployment, it is desirable to understand the optimal trade-offs between on-device latency and model accuracy. In this work, we leverage recent methodological advances in Bayesian optimization over high-dimensional search spaces and multi-objective Bayesian optimization to efficiently explore these trade-offs for a production-scale on-device natural language understanding model at Facebook.