Local Thresholding in General Network Graphs

TL;DR

This paper introduces a novel local thresholding algorithm that operates effectively on general network graphs without requiring cycle-free routing, expanding applicability to real-world distributed systems.

Contribution

It presents the first cycle-free thresholding algorithm for general graphs, utilizing new weighted vector methods, stopping rules, and balance correction techniques.

Findings

Validated on networks of up to 80,000 peers

Works across Internet, peer-to-peer, and sensor network topologies

Maintains calculation capabilities of previous algorithms without cycle restrictions

Abstract

Local thresholding algorithms were first presented more than a decade ago and have since been applied to a variety of data mining tasks in peer-to-peer systems, wireless sensor networks, and in grid systems. One critical assumption made by those algorithms has always been cycle-free routing. The existence of even one cycle may lead all peers to the wrong outcome. Outside the lab, unfortunately, cycle freedom is not easy to achieve. This work is the first to lift the requirement of cycle freedom by presenting a local thresholding algorithm suitable for general network graphs. The algorithm relies on a new repositioning of the problem in weighted vector arithmetics, on a new stopping rule, whose proof does not require that the network be cycle free, and on new methods for balance correction when the stopping rule fails. The new stopping and update rules permit calculation of the very same functions that were calculable using previous algorithms, which do assume cycle freedom. The algorithm is implemented on a standard peer-to-peer simulator and is validated for networks of up to 80,000 peers, organized in three different topologies, which are representative of the topology of major current distributed systems: the Internet, structured peer-to-peer systems, and wireless sensor networks.