Demystifying Map Space Exploration for NPUs

TL;DR

This paper systematically compares different search techniques for mapping neural networks on accelerators, providing insights into their navigation of map space and introducing two new techniques that improve mapping efficiency and robustness.

Contribution

It offers the first systematic comparison of search techniques for map space exploration and proposes two novel methods to enhance existing mappers.

Findings

New techniques demonstrate speedups and scalability.

Insights into how search methods navigate map space.

Enhanced robustness across diverse DNN models.

Abstract

Map Space Exploration is the problem of finding optimized mappings of a Deep Neural Network (DNN) model on an accelerator. It is known to be extremely computationally expensive, and there has been active research looking at both heuristics and learning-based methods to make the problem computationally tractable. However, while there are dozens of mappers out there (all empirically claiming to find better mappings than others), the research community lacks systematic insights on how different search techniques navigate the map-space and how different mapping axes contribute to the accelerator's performance and efficiency. Such insights are crucial to developing mapping frameworks for emerging DNNs that are increasingly irregular (due to neural architecture search) and sparse, making the corresponding map spaces much more complex. In this work, rather than proposing yet another mapper, we do a first-of-its-kind apples-to-apples comparison of search techniques leveraged by different mappers. Next, we extract the learnings from our study and propose two new techniques that can augment existing mappers -- warm-start and sparsity-aware -- that demonstrate speedups, scalability, and robustness across diverse DNN models.