Ira: Efficient Transaction Replay for Distributed Systems

TL;DR

Ira is a framework that accelerates transaction replay in distributed systems by transmitting compact hints, significantly reducing backup replay time as demonstrated in Ethereum case studies.

Contribution

The paper introduces Ira, a novel hint-based approach to improve transaction replay efficiency, with a concrete protocol Ira-L for Ethereum that achieves substantial speedups.

Findings

Ira-L reduces Ethereum backup replay time from 6.5 hours to 16 minutes.

Hints add about 5% of block payload and impose a 10.9% execution overhead.

Median per-block speedup of 25x over baseline Ethereum client reth.

Abstract

In primary-backup replication, consensus latency is bounded by the time for backup nodes to replay (re-execute) transactions proposed by the primary. In this work, we present Ira, a framework to accelerate backup replay by transmitting compact \emph{hints} alongside transaction batches. Our key insight is that the primary, having already executed transactions, possesses knowledge of future access patterns which is exactly the information needed for optimal replay. We use Ethereum for our case study and present a concrete protocol, Ira-L, within our framework to improve cache management of Ethereum block execution. The primaries implementing Ira-L provide hints that consist of the working set of keys used in an Ethereum block and one byte of metadata per key indicating the table to read from, and backups use these hints for efficient block replay. We evaluated Ira-L against the state-of-the-art Ethereum client reth over two weeks of Ethereum mainnet activity ($100,800$ blocks containing over $24$ million transactions). Our hints are compact, adding a median of $47$ KB compressed per block ($\sim5\%$ of block payload). We observe that the sequential hint generation and block execution imposes a $28.6\%$ wall-time overhead on the primary, though the direct cost from hints is $10.9\%$ of execution time; all of which can be pipelined and parallelized in production deployments. On the backup side, we observe that Ira-L achieves a median per-block speedup of $25\times$ over baseline reth. With $16$ prefetch threads, aggregate replay time drops from $6.5$ hours to $16$ minutes ($23.6\times$ wall-time speedup).