Minimize Your Critical Path with Combine-and-Exchange Locks

TL;DR

This paper introduces Combine-and-Exchange Scheduling (CES), a novel approach for userspace synchronization that reduces delays and improves throughput for coroutine-based applications by keeping critical sections on the same thread.

Contribution

We propose CES, a new scheduling method that minimizes critical path delays in userspace synchronization for coroutines and fibers, leading to significant performance gains.

Findings

3-fold performance improvements in application benchmarks

8-fold performance improvements in microbenchmarks

Applicable to many existing languages and libraries

Abstract

Coroutines are experiencing a renaissance as many modern programming languages support the use of cooperative multitasking for highly parallel or asynchronous applications. One of the greatest advantages of this is that concurrency and synchronization is manged entirely in the userspace, omitting heavy-weight system calls. However, we find that state-of-the-art userspace synchronization primitives approach synchronization in the userspace from the perspective of kernel-level scheduling. This introduces unnecessary delays on the critical path of the application, limiting throughput. In this paper, we re-think synchronization for tasks that are scheduled entirely in the userspace (e.g., coroutines, fibers, etc.). We develop Combine-and-Exchange Scheduling (CES), a novel scheduling approach that ensures contended critical sections stay on the same thread of execution while parallelizable work is evenly spread across the remaining threads. We show that our approach can be applied to many existing languages and libraries, resulting in 3-fold performance improvements in application benchmarks as well as 8-fold performance improvements in microbenchmarks.