Low-Latency, High-Throughput Garbage Collection (Extended Version)

TL;DR

LXR is a novel garbage collector that significantly improves responsiveness and throughput by minimizing copying, using reference counting, and introducing a low-overhead write barrier, outperforming existing collectors like Shenandoah.

Contribution

LXR introduces a new approach combining reference counting, limited copying, and a novel write barrier to optimize garbage collection responsiveness and throughput.

Findings

LXR achieves 6x higher throughput on Lucene.

LXR delivers 30x lower tail latency compared to Shenandoah.

LXR effectively balances responsiveness and memory reclamation efficiency.

Abstract

Production garbage collectors make substantial compromises in pursuit of reduced pause times. They require far more CPU cycles and memory than prior simpler collectors. concurrent copying collectors (C4, ZGC, and Shenandoah) suffer from the following design limitations. 1) Concurrent copying. They only reclaim memory by copying, which is inherently expensive with high memory bandwidth demands. Concurrent copying also requires expensive read and write barriers. 2) Scalability. They depend on tracing, which in the limit and in practice does not scale. 3) Immediacy. They do not reclaim older objects promptly, incurring high memory overheads. We present LXR, which takes a very different approach to optimizing responsiveness and throughput by minimizing concurrent collection work and overheads. 1) LXR reclaims most memory without any copying by using the Immix heap structure. It then combats fragmentation with limited judicious stop-the-world copying. 2) LXR uses reference counting to achieve both scalability and immediacy, promptly reclaiming young and old objects. It uses concurrent tracing as needed for identifying cyclic garbage. 3) To minimize pause times while allowing judicious copying of mature objects, LXR introduces remembered sets for reference counting and concurrent decrement processing. 4) LXR introduces a novel low-overhead write barrier that combines coalescing reference counting, concurrent tracing, and remembered set maintenance. The result is a collector with excellent responsiveness and throughput. On the widely-used Lucene search engine with a generously sized heap, LXR has 6x higher throughput while delivering 30x lower 99.9 percentile tail latency than the popular Shenandoah production collector in its default configuration.