Coflex: Enhancing HW-NAS with Sparse Gaussian Processes for Efficient and Scalable DNN Accelerator Design

TL;DR

Coflex introduces a scalable HW-NAS framework using Sparse Gaussian Processes to efficiently optimize neural network architectures for edge accelerators, significantly reducing computation while maintaining high accuracy.

Contribution

It integrates Sparse Gaussian Processes with Bayesian optimization to address scalability issues in HW-NAS, enabling efficient large-scale search space exploration.

Findings

Outperforms state-of-the-art in accuracy and energy-delay-product.

Achieves 1.9x to 9.5x speed-up in search process.

Effectively scales to large search spaces without loss of performance.

Abstract

Hardware-Aware Neural Architecture Search (HW-NAS) is an efficient approach to automatically co-optimizing neural network performance and hardware energy efficiency, making it particularly useful for the development of Deep Neural Network accelerators on the edge. However, the extensive search space and high computational cost pose significant challenges to its practical adoption. To address these limitations, we propose Coflex, a novel HW-NAS framework that integrates the Sparse Gaussian Process (SGP) with multi-objective Bayesian optimization. By leveraging sparse inducing points, Coflex reduces the GP kernel complexity from cubic to near-linear with respect to the number of training samples, without compromising optimization performance. This enables scalable approximation of large-scale search space, substantially decreasing computational overhead while preserving high predictive accuracy. We evaluate the efficacy of Coflex across various benchmarks, focusing on accelerator-specific architecture. Our experimental results show that Coflex outperforms state-of-the-art methods in terms of network accuracy and Energy-Delay-Product, while achieving a computational speed-up ranging from 1.9x to 9.5x.