Fault-Tolerant Aggregation: Flow-Updating Meets Mass-Distribution

TL;DR

This paper introduces MDFU, a fault-tolerant distributed aggregation protocol combining Flow-Updating with Mass-Distribution, providing resilience to message loss and demonstrating efficient convergence and accuracy in practical scenarios.

Contribution

The paper presents MDFU, the first FU-based protocol for distributed aggregation with proven convergence under message loss, and introduces MDFU-LP for improved accuracy.

Findings

MDFU converges efficiently despite message loss.

MDFU-LP maintains high accuracy under high message loss.

Experimental results confirm practical robustness of both protocols.

Abstract

Flow-Updating (FU) is a fault-tolerant technique that has proved to be efficient in practice for the distributed computation of aggregate functions in communication networks where individual processors do not have access to global information. Previous distributed aggregation protocols, based on repeated sharing of input values (or mass) among processors, sometimes called Mass-Distribution (MD) protocols, are not resilient to communication failures (or message loss) because such failures yield a loss of mass. In this paper, we present a protocol which we call Mass-Distribution with Flow-Updating (MDFU). We obtain MDFU by applying FU techniques to classic MD. We analyze the convergence time of MDFU showing that stochastic message loss produces low overhead. This is the first convergence proof of an FU-based algorithm. We evaluate MDFU experimentally, comparing it with previous MD and FU protocols, and verifying the behavior predicted by the analysis. Finally, given that MDFU incurs a fixed deviation proportional to the message-loss rate, we adjust the accuracy of MDFU heuristically in a new protocol called MDFU with Linear Prediction (MDFU-LP). The evaluation shows that both MDFU and MDFU-LP behave very well in practice, even under high rates of message loss and even changing the input values dynamically.