MAPA: Multi-Accelerator Pattern Allocation Policy for Multi-Tenant GPU Servers

TL;DR

This paper introduces MAPA, a graph pattern mining approach for efficient multi-accelerator workload allocation in multi-tenant GPU servers, improving execution times and communication efficiency.

Contribution

MAPA provides a novel generalized allocation policy tailored for complex multi-accelerator topologies and communication patterns, outperforming baseline policies.

Findings

12.4% speedup for 75th percentile jobs

up to 35% reduction in worst-case execution time

improved inter-accelerator communication efficiency

Abstract

Multi-accelerator servers are increasingly being deployed in shared multi-tenant environments (such as in cloud data centers) in order to meet the demands of large-scale compute-intensive workloads. In addition, these accelerators are increasingly being inter-connected in complex topologies and workloads are exhibiting a wider variety of inter-accelerator communication patterns. However, existing allocation policies are ill-suited for these emerging use-cases. Specifically, this work identifies that multi-accelerator workloads are commonly fragmented leading to reduced bandwidth and increased latency for inter-accelerator communication. We propose Multi-Accelerator Pattern Allocation (MAPA), a graph pattern mining approach towards providing generalized allocation support for allocating multi-accelerator workloads on multi-accelerator servers. We demonstrate that MAPA is able to improve the execution time of multi-accelerator workloads and that MAPA is able to provide generalized benefits across various accelerator topologies. Finally, we demonstrate a speedup of 12.4% for 75th percentile of jobs with the worst case execution time reduced by up to 35% against baseline policy using MAPA.