Data Aware Differentiable Neural Architecture Search for Tiny Keyword Spotting Applications

TL;DR

This paper introduces a novel data-aware neural architecture search method that co-optimizes model architecture and data parameters to create efficient, accurate TinyML keyword spotting systems, reducing complexity and resource use.

Contribution

It extends differentiable neural architecture search by including data configuration parameters, enabling joint optimization for TinyML applications.

Findings

Generated lean, highly accurate keyword spotting models

Demonstrated resource-performance balance in TinyML systems

Showed effectiveness of data-aware search in reducing complexity

Abstract

The success of Machine Learning is increasingly tempered by its significant resource footprint, driving interest in efficient paradigms like TinyML. However, the inherent complexity of designing TinyML systems hampers their broad adoption. To reduce this complexity, we introduce "Data Aware Differentiable Neural Architecture Search". Unlike conventional Differentiable Neural Architecture Search, our approach expands the search space to include data configuration parameters alongside architectural choices. This enables Data Aware Differentiable Neural Architecture Search to co-optimize model architecture and input data characteristics, effectively balancing resource usage and system performance for TinyML applications. Initial results on keyword spotting demonstrate that this novel approach to TinyML system design can generate lean but highly accurate systems.