MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores

TL;DR

This paper introduces MAGMA, an optimization framework designed to efficiently map multiple DNNs onto multi-core accelerators, improving resource utilization and performance through a structured search approach.

Contribution

The paper presents MAGMA, a novel optimization algorithm with custom operators that effectively explores the large search space for multi-DNN mapping on accelerators.

Findings

MAGMA outperforms state-of-the-art methods in mapping quality.

It achieves better resource utilization across various accelerator configurations.

MAGMA demonstrates consistent improvements in mapping efficiency.

Abstract

As Deep Learning continues to drive a variety of applications in edge and cloud data centers, there is a growing trend towards building large accelerators with several sub-accelerator cores/chiplets. This work looks at the problem of supporting multi-tenancy on such accelerators. In particular, we focus on the problem of mapping jobs from several DNNs simultaneously on an accelerator. Given the extremely large search space, we formulate the search as an optimization problem and develop an optimization framework called M3E. In addition, we develop a specialized optimization algorithm called MAGMA with custom operators to enable structured sample-efficient exploration. We quantitatively compare MAGMA with several state-of-the-art methods, black-box optimization, and reinforcement learning methods across different accelerator settings (large/small accelerators) and different sub-accelerator configurations (homogeneous/heterogeneous), and observe MAGMA can consistently find better mappings.