Flex-MIG: Enabling Distributed Execution on MIG

TL;DR

Flex-MIG is a software framework that enhances GPU cluster utilization by enabling flexible resource sharing among NVIDIA MIG instances, reducing fragmentation and improving workload completion times without hardware changes.

Contribution

It introduces a novel software-only approach that replaces rigid hardware allocation with a flexible one-to-many model and supports shared-memory collectives across MIGs.

Findings

Reduces cluster fragmentation significantly.

Improves makespan by up to 17%.

Eliminates reconfiguration drain requirements.

Abstract

GPU clusters in multi-tenant settings often suffer from underutilization, making GPU-sharing technologies essential for efficient resource use. Among them, NVIDIA Multi-Instance GPU (MIG) has gained traction for providing hardware-level isolation that enables concurrent workloads without interference. However, MIG's hardware rigidity and the conventional one-to-one allocation model jointly lead to severe fragmentation and cluster-wide underutilization. We present Flex-MIG, a software-only framework that replaces one-to-one with a one-to-many allocation model and enables host-shared-memory collectives across MIG instances without hardware modification. Flex-MIG eliminates drain-required reconfiguration, reduces fragmentation, and improves makespan by up to 17% across diverse traces, showing that rethinking MIG's operational model as a software-coordinated layer substantially improves cluster efficiency.