Lightweight User-Space Record And Replay

TL;DR

This paper presents RR, a low-overhead, user-space record-and-replay system that operates on stock hardware and Linux, enabling practical reverse debugging without heavy virtualization or kernel modifications.

Contribution

It introduces a novel user-space approach to record-and-replay that works on modern hardware and OS, avoiding traditional heavyweight methods.

Findings

RR records and replays real-world workloads with low overhead

The system is implemented entirely in user space on stock hardware

RR is used as an open-source reverse debugger in practice

Abstract

The ability to record and replay program executions with low overhead enables many applications, such as reverse-execution debugging, debugging of hard-to-reproduce test failures, and "black box" forensic analysis of failures in deployed systems. Existing record-and-replay approaches rely on recording an entire virtual machine (which is heavyweight), modifying the OS kernel (which adds deployment and maintenance costs), or pervasive code instrumentation (which imposes significant performance and complexity overhead). We investigated whether it is possible to build a practical record-and-replay system avoiding all these issues. The answer turns out to be yes --- if the CPU and operating system meet certain non-obvious constraints. Fortunately modern Intel CPUs, Linux kernels and user-space frameworks meet these constraints, although this has only become true recently. With some novel optimizations, our system RR records and replays real-world workloads with low overhead with an entirely user-space implementation running on stock hardware and operating systems. RR forms the basis of an open-source reverse-execution debugger seeing significant use in practice. We present the design and implementation of RR, describe its performance on a variety of workloads, and identify constraints on hardware and operating system design required to support our approach.