Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration

TL;DR

This paper introduces HDX, a method that enables differentiable neural architecture and hardware co-exploration to satisfy hard constraints like frame rate without sacrificing overall design goals.

Contribution

HDX is the first approach to systematically incorporate hard constraints into differentiable co-exploration, improving solution quality and constraint satisfaction.

Findings

HDX effectively satisfies hard constraints such as frame rate.

HDX maintains high search efficiency comparable to unconstrained methods.

HDX produces high-quality neural architectures and accelerators that meet specified constraints.

Abstract

Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.