Grassroots Operator Search for Model Edge Adaptation

TL;DR

This paper introduces Grassroots Operator Search (GOS), a novel HW-NAS method that adaptively replaces operators in models with mathematically defined efficient alternatives, significantly improving speed on edge devices without sacrificing accuracy.

Contribution

GOS provides a mathematically grounded, flexible approach for operator replacement in neural networks, enabling efficient edge device deployment without relying on traditional macro-architecture search.

Findings

Achieves at least 2.2x speedup on edge devices while maintaining accuracy.

Outperforms original models in various DL architectures on Redmi Note 7S and Raspberry Pi3.

Demonstrates state-of-the-art pulse rate estimation with reduced computational complexity.

Abstract

Hardware-aware Neural Architecture Search (HW-NAS) is increasingly being used to design efficient deep learning architectures. An efficient and flexible search space is crucial to the success of HW-NAS. Current approaches focus on designing a macro-architecture and searching for the architecture's hyperparameters based on a set of possible values. This approach is biased by the expertise of deep learning (DL) engineers and standard modeling approaches. In this paper, we present a Grassroots Operator Search (GOS) methodology. Our HW-NAS adapts a given model for edge devices by searching for efficient operator replacement. We express each operator as a set of mathematical instructions that capture its behavior. The mathematical instructions are then used as the basis for searching and selecting efficient replacement operators that maintain the accuracy of the original model while reducing computational complexity. Our approach is grassroots since it relies on the mathematical foundations to construct new and efficient operators for DL architectures. We demonstrate on various DL models, that our method consistently outperforms the original models on two edge devices, namely Redmi Note 7S and Raspberry Pi3, with a minimum of 2.2x speedup while maintaining high accuracy. Additionally, we showcase a use case of our GOS approach in pulse rate estimation on wristband devices, where we achieve state-of-the-art performance, while maintaining reduced computational complexity, demonstrating the effectiveness of our approach in practical applications.