Nixie: Efficient, Transparent Temporal Multiplexing for Consumer GPUs
Yechen Xu, Yifei Wang, Nathanael Ren, Yiran Chen, Danyang Zhuo

TL;DR
Nixie is a system that improves GPU memory utilization and responsiveness for consumer GPUs running large ML workloads by enabling efficient temporal multiplexing without requiring application or driver modifications.
Contribution
Nixie introduces a novel system for transparent temporal multiplexing on consumer GPUs, optimizing memory and latency without application changes.
Findings
Up to 3.8x latency improvement for interactive tasks
Reduces CPU pinned memory usage by 66.8%
Enhances GPU resource utilization efficiency
Abstract
Consumer machines are increasingly running large ML workloads such as large language models (LLMs), text-to-image generation, and interactive image editing. Unlike datacenter GPUs, consumer GPUs serve single-user, rapidly changing workloads, and each model's working set often nearly fills the GPU memory. As a result, existing sharing mechanisms (e.g., NVIDIA Unified Virtual Memory) perform poorly due to memory thrashing and excessive use of CPU pinned memory when multiple applications are active. We design and implement Nixie, a system that enables efficient and transparent temporal multiplexing on consumer GPUs without requiring any application or driver changes. Nixie is a system service that coordinates GPU memory allocation and kernel launch behavior to efficiently utilize the CPU-GPU bi-directional bandwidth and CPU pinned memory. A lightweight scheduler in Nixie further improves…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Cloud Computing and Resource Management · Big Data and Digital Economy
