Towards a Performance Model for Byzantine Fault Tolerant (Storage) Services

TL;DR

This paper develops the first performance model for PBFT considering unreliable channels and transport protocols, validated through simulations and real implementation comparisons, revealing network layer design impacts on practical performance.

Contribution

It introduces a performance model for PBFT that accounts for unreliable channels and transport protocols, supported by extensive simulations and real-world measurements.

Findings

UDP can significantly speed up PBFT over lossy channels.

Network layer design impacts PBFT performance and offers optimization opportunities.

Simulation results align with real implementation measurements.

Abstract

Byzantine fault-tolerant systems have been researched for more than four decades, and although shown possible early, the solutions were impractical for a long time. With PBFT the first practical solution was proposed in 1999 and spawned new research which culminated in novel applications using it today. Although the safety and liveness properties of PBFT-type protocols have been rigorously analyzed, when it comes to practical performance only empirical results - often in artificial settings - are known and imperfections on the communication channels are not specifically considered. In this work we present the first performance model for PBFT specifically considering the impact of unreliable channels and the use of different transport protocols over them. We also did extensive simulations to verify the model and to gain more insight on the impact of deployment parameters on the overall transaction time. We show that the usage of UDP can lead to significant speedup for PBFT protocols compared to TCP when tuned accordingly even over lossy channels. Finally, we compared the simulation to a real implementation and measure the benefits of a developed improvement directly. We found that the impact on the design of the network layer has been overlooked in the past but offers some additional room for improvement when it comes to practical performance. In this work we are focusing on the optimistic case with no node failures, as this is hopefully the most relevant situation.