Scalable Performance Evaluation of Byzantine Fault-Tolerant Systems Using Network Simulation

TL;DR

This paper presents a network simulation approach for scalable performance evaluation of Byzantine fault-tolerant systems, enabling accurate, cost-effective analysis without modifying existing implementations.

Contribution

It introduces a plug-and-play simulation architecture for BFT protocols that accurately predicts performance at large scale, reducing reliance on costly real-world deployments.

Findings

Simulations closely match real system performance at large scale.

Network becomes the main performance bottleneck in large systems.

Evaluation of modern BFT protocols like HotStuff and Kauri in realistic scenarios.

Abstract

Recent Byzantine fault-tolerant (BFT) state machine replication (SMR) protocols increasingly focus on scalability to meet the requirements of distributed ledger technology (DLT). Validating the performance of scalable BFT protocol implementations requires careful evaluation. Our solution uses network simulations to forecast the performance of BFT protocols while experimentally scaling the environment. Our method seamlessly plug-and-plays existing BFT implementations into the simulation without requiring code modification or re-implementation, which is often time-consuming and error-prone. Furthermore, our approach is also significantly cheaper than experiments with real large-scale cloud deployments. In this paper, we first explain our simulation architecture, which enables scalable performance evaluations of BFT systems through high performance network simulations. We validate the accuracy of these simulations for predicting the performance of BFT systems by comparing simulation results with measurements of real systems deployed on cloud infrastructures. We found that simulation results display a reasonable approximation at a larger system scale, because the network eventually becomes the dominating factor limiting system performance. In the second part of our paper, we use our simulation method to evaluate the performance of PBFT and BFT protocols from the blockchain generation, such as HotStuff and Kauri, in large-scale and realistic wide-area network scenarios, as well as under induced faults.