Range-Based Set Reconciliation via Range-Summarizable Order-Statistics Stores

TL;DR

This paper introduces a new storage abstraction called range-summarizable order-statistics store (RSOS) for efficient range-based set reconciliation, and demonstrates its implementation and benefits within a persistent memory engine.

Contribution

It formalizes the RSOS abstraction, shows how to implement it with augmented B+-trees, and develops AELMDB, a persistent memory system that reduces reconciliation costs.

Findings

RSOS enables efficient recursive range refinement in set reconciliation.

Augmented B+-trees can realize RSOS with bounded local work.

AELMDB significantly reduces reconciliation workload in experiments.

Abstract

Range-Based Set Reconciliation (RBSR) synchronizes ordered sets by recursively comparing summaries of contiguous ranges and refining only the mismatching parts. While its communication complexity is well understood, its local computational cost fundamentally depends on the storage backend that must answer repeated range-summary, rank, and enumeration queries during refinement. We argue that a natural storage abstraction for RBSR implementations based on composable range aggregates is a \emph{range-summarizable order-statistics store} (RSOS): a dynamic ordered-set structure supporting composable summaries of contiguous ranges together with rank/select navigation. This identifies and formalizes the backend contract needed for efficient recursive refinement, combining range-summary support with order-statistics navigation for balanced partitioning. We then show that a specific augmentation of B\textsuperscript{+}-trees with subtree counts and composable summaries realizes a RSOS, and we derive corresponding bounds on local reconciliation work in this abstract storage model. Finally, we introduce AELMDB, an extension of LMDB that realizes this design inside a persistent memory-mapped engine, and evaluate it through an integration with Negentropy. The results show that placing the reconciliation oracle inside the storage tree substantially reduces local reconciliation cost on the evaluated reconciliation-heavy workloads compared with an open-source persistent baseline based on auxiliary tree caches, while the window-subrange ablation further confirms the usefulness of the systems optimizations built on top of the core aggregate representation.