Carbon-Efficient Neural Architecture Search

TL;DR

This paper introduces CE-NAS, a novel neural architecture search framework that optimizes for both model performance and carbon efficiency by balancing energy consumption during the search process.

Contribution

It proposes a multi-objective NAS framework that dynamically balances energy-efficient sampling and evaluation, improving carbon footprint and search efficiency.

Findings

CE-NAS outperforms baseline methods in carbon efficiency.

Trace-driven simulations validate improved search efficiency.

The framework adapts to current carbon emissions for optimized performance.

Abstract

This work presents a novel approach to neural architecture search (NAS) that aims to reduce energy costs and increase carbon efficiency during the model design process. The proposed framework, called carbon-efficient NAS (CE-NAS), consists of NAS evaluation algorithms with different energy requirements, a multi-objective optimizer, and a heuristic GPU allocation strategy. CE-NAS dynamically balances energy-efficient sampling and energy-consuming evaluation tasks based on current carbon emissions. Using a recent NAS benchmark dataset and two carbon traces, our trace-driven simulations demonstrate that CE-NAS achieves better carbon and search efficiency than the three baselines.