SGPRS: Seamless GPU Partitioning Real-Time Scheduler for Periodic Deep Learning Workloads

TL;DR

This paper introduces SGPRS, a novel real-time GPU scheduler that enables seamless partition switching for deep learning workloads, improving deadline adherence and overall GPU utilization.

Contribution

SGPRS is the first scheduler to consider zero-configuration partition switching in real-time GPU scheduling for deep neural network workloads.

Findings

SGPRS improves deadline satisfaction for parallel GPU tasks.

It sustains performance beyond the typical resource pivot point.

The scheduler enables more efficient GPU utilization in real-time DNN applications.

Abstract

Deep Neural Networks (DNNs) are useful in many applications, including transportation, healthcare, and speech recognition. Despite various efforts to improve accuracy, few works have studied DNN in the context of real-time requirements. Coarse resource allocation and sequential execution in existing frameworks result in underutilization. In this work, we conduct GPU speedup gain analysis and propose SGPRS, the first real-time GPU scheduler considering zero configuration partition switch. The proposed scheduler not only meets more deadlines for parallel tasks but also sustains overall performance beyond the pivot point.