A Domain Specific Language for Testing Consensus Implementations

TL;DR

This paper introduces Netrix, a domain-specific language and tool for testing distributed consensus implementations, improving coverage, bug reproduction, and regression testing, demonstrated on blockchain and consensus protocols.

Contribution

The paper presents a novel testing methodology and tool, Netrix, tailored for consensus algorithms, enabling better coverage, bug detection, and regression testing in distributed systems.

Findings

Identified 4 deviations in Tendermint from its protocol specification.

Reproduced 4 known bugs in Raft.

Validated the effectiveness of Netrix on multiple consensus protocols.

Abstract

Large-scale, fault-tolerant, distributed systems are the backbone for many critical software services. Since they must execute correctly in a possibly adversarial environment with arbitrary communication delays and failures, the underlying algorithms are intricate. In particular, achieving consistency and data retention relies on intricate consensus (state machine replication) protocols. Ensuring the reliability of implementations of such protocols remains a significant challenge because of the enormous number of exceptional conditions that may arise in production. We propose a methodology and a tool called Netrix for testing such implementations that aims to exploit programmer's knowledge to improve coverage, enables robust bug reproduction, and can be used in regression testing across different versions of an implementation. As evaluation, we apply our tool to a popular proof of stake blockchain protocol, Tendermint, which relies on a Byzantine consensus algorithm, a benign consensus algorithm, Raft, and BFT-Smart. We were able to identify 4 deviations of the Tendermint implementation from the protocol specification and check their absence on an updated implementation. Additionally, we were able to reproduce 4 previously known bugs in Raft.