Towards Fast, Adaptive, and Hardware-Assisted User-Space Scheduling

TL;DR

This paper introduces LibPreemptible, a user-space threading library that leverages new hardware features to improve tail latency and throughput in datacenter applications without kernel modifications.

Contribution

It presents a novel, hardware-assisted, adaptive user-space scheduling system that outperforms existing solutions like Shinjuku in tail latency and throughput.

Findings

Significant tail latency reduction compared to prior systems

Improved throughput for microsecond-scale workloads

Flexible scheduling policies adaptable to varying loads

Abstract

Modern datacenter applications are prone to high tail latencies since their requests typically follow highly-dispersive distributions. Delivering fast interrupts is essential to reducing tail latency. Prior work has proposed both OS- and system-level solutions to reduce tail latencies for microsecond-scale workloads through better scheduling. Unfortunately, existing approaches like customized dataplane OSes, require significant OS changes, experience scalability limitations, or do not reach the full performance capabilities hardware offers. The emergence of new hardware features like UINTR exposed new opportunities to rethink the design paradigms and abstractions of traditional scheduling systems. We propose LibPreemptible, a preemptive user-level threading library that is flexible, lightweight, and adaptive. LibPreemptible was built with a set of optimizations like LibUtimer for scalability, and deadline-oriented API for flexible policies, time-quantum controller for adaptiveness. Compared to the prior state-of-the-art scheduling system Shinjuku, our system achieves significant tail latency and throughput improvements for various workloads without modifying the kernel. We also demonstrate the flexibility of LibPreemptible across scheduling policies for real applications experiencing varying load levels and characteristics.