No free lunch for avoiding clustering vulnerabilities in distributed systems

TL;DR

This paper explores how clustering in complex systems can lead to design failures and proposes a framework to manage these vulnerabilities using insights from statistical physics.

Contribution

The paper introduces a novel framework to identify and quantify trade-offs between clustering and uncertainty in design objectives using statistical physics.

Findings

Heterogeneous networks can exhibit repulsion-driven clustering in addition to attraction-driven clustering.

Trade-offs between clustering and uncertainty in design objectives are common in distributed systems.

The framework connects naval engineering models to entropy-driven phenomena in nanoscale self-assembly.

Abstract

Emergent design failures are ubiquitous in complex systems, and often arise when system elements cluster. Approaches to systematically reduce clustering could improve a design’s resilience, but reducing clustering is difficult if it is driven by collective interactions among design elements. Here, we use techniques from statistical physics to identify mechanisms by which spatial clusters of design elements emerge in complex systems modelled by heterogeneous networks. We find that, in addition to naive, attraction-driven clustering, heterogeneous networks can exhibit emergent, repulsion-driven clustering. We draw quantitative connections between our results on a model system in naval engineering to entropy-driven phenomena in nanoscale self-assembly, and give a general argument that the clustering phenomena we observe should arise in many distributed systems. We identify circumstances under which generic design problems will exhibit trade-offs between clustering and uncertainty in design objectives, and we present a framework to identify and quantify trade-offs to manage clustering vulnerabilities.

Figures