A comprehensive evaluation of spatial co-execution on GPUs using MPS and MIG technologies

TL;DR

This paper evaluates NVIDIA's MPS and MIG GPU technologies, analyzing their trade-offs and performance impacts for spatial co-execution of multiple applications.

Contribution

It provides a comprehensive comparison of MPS and MIG, revealing their strengths, limitations, and trade-offs for optimizing GPU resource sharing.

Findings

MPS can improve performance by up to 30% and reduce energy consumption by 20%.

MPS performance degrades by 30% under memory contention.

MIG offers consistent improvements with better memory contention handling but has higher overhead.

Abstract

To mitigate the increasingly common underutilization of computational resources in modern GPUs, spatial sharing methods enable multiple applications to use them simultaneously. This work presents a comprehensive evaluation of NVIDIA's primary technologies to achieve that goal: Multi-Process Service (MPS) and Multi-Instance GPU (MIG). Our findings reveal a crucial trade-off between MPS's flexibility and MIG's isolation, and provide many key insights for improving the co-execution strategy according to job profiles. In the most favorable scenarios, MPS improves performance by up to 30% and reduces energy by about 20%, using its provisioning option to avoid resource monopolization. However, under memory contention, it suffers severe degradation, worsening performance by around 30%. Conversely, MIG's full hardware isolation resolves memory contention, leading to more consistent improvements, but these gains are tempered by higher overhead, and its rigid scheme can degrade performance in certain cases.