An Effectively $\Omega(c)$ Language and Runtime

TL;DR

This paper proposes a new language and runtime designed to guarantee effectively constant performance and memory overhead, aiming to improve tail latency and predictability in application responsiveness.

Contribution

It introduces a language and runtime framework that ensures $oldsymbol{ ext{Omega}(c)}$ performance, providing constant-time operations and bounded pauses, addressing tail latency issues.

Findings

Design achieves effectively constant time operations.

Memory overhead remains fixed and predictable.

Garbage collection has bounded pauses.

Abstract

The performance of an application/runtime is usually thought of as a continuous function where, the lower the amount of memory/time used on a given workload, then the better the compiler/runtime is. However, in practice, good performance of an application is conceptually more of a binary function -- either the application responds in under, say 100ms, and is fast enough for a user to barely notice, or it takes a noticeable amount of time, leaving the user waiting and potentially abandoning the task. Thus, performance really means how often the application is fast enough to be usable, leading industrial developers to focus on the 95th and 99th percentile latencies as heavily, or moreso, than average response time. Unfortunately, tracking and optimizing for these high percentile latencies is difficult and often requires a deep understanding of the application, runtime, GC, and OS interactions. This is further complicated by the fact that tail performance is often only seen occasionally, and is specific to a certain workload or input, making these issues uniquely painful to handle. Our vision is to create a language and runtime that is designed to be $\Omega(c)$ in its performance -- that is, it is designed to have an effectively constant time to execute all operations, there is a constant fixed memory overhead for the application footprint, and the garbage-collector performs a constant amount of work per allocation + a (small) bounded pause for all collection/release operations.