Hybrid Transactional Replication: State-Machine and Deferred-Update Replication Combined

TL;DR

This paper introduces Hybrid Transactional Replication (HTR), combining optimistic deferred update and deterministic state machine modes, enhanced by machine learning for adaptive performance, ensuring strong consistency and improved throughput.

Contribution

The paper presents HTR, a novel replication scheme that integrates two modes with an adaptive oracle, maintaining strong guarantees while optimizing performance.

Findings

Up to 50% throughput improvement with ML-based oracle

HTR maintains strong consistency guarantees

Effective adaptation to changing workloads

Abstract

We propose Hybrid Transactional Replication (HTR), a novel replication scheme for highly dependable services. It combines two schemes: a transaction is executed either optimistically by only one service replica in the deferred update mode (DU), or deterministically by all replicas in the state machine mode (SM); the choice is made by an oracle. The DU mode allows for parallelism and thus takes advantage of multicore hardware. In contrast to DU, the SM mode guarantees abort-free execution, so it is suitable for irrevocable operations and transactions generating high contention. For expressiveness, transactions can be discarded or retried on demand. We formally prove that the higher flexibility of the scheme does not come at the cost of weaker guarantees for clients: HTR satisfies strong consistency guarantees akin to those provided by other popular transactional replication schemes such as Deferred Update Replication. We developed HTR-enabled Paxos STM, an object-based distributed transactional memory system, and evaluated it thoroughly under various workloads. We show the benefits of using a novel oracle, which relies on machine learning techniques for automatic adaptation to changing conditions. In our tests, the ML-based oracle provides up to 50% improvement in throughput when compared to the system running with DU-only or SM-only oracles. Our approach is inspired by a well known algorithm used in the context of the multi-armed bandit problem.