No Free Lunch for Avoiding Clustering Vulnerabilities in Distributed Systems

TL;DR

This paper investigates how clustering in complex distributed systems can be driven by attraction or repulsion mechanisms, using statistical physics to identify trade-offs and propose a framework for managing clustering vulnerabilities.

Contribution

It introduces a physics-inspired framework to analyze and quantify clustering mechanisms in heterogeneous networks, revealing new repulsion-driven clustering phenomena.

Findings

Heterogeneous networks can exhibit repulsion-driven clustering.

A quantitative connection to nanoscale self-assembly phenomena.

A framework to identify trade-offs between clustering and design uncertainty.

Abstract

Emergent design failures are ubiquitous in complex systems, and often arise when system elements cluster. Approaches to systematically reduce clustering could improve a design's resilience, but reducing clustering is difficult if it is driven by collective interactions among design elements. Here, we use techniques from statistical physics to identify mechanisms by which spatial clusters of design elements emerge in complex systems modelled by heterogeneous networks. We find that, in addition to naive, attraction-driven clustering, heterogeneous networks can exhibit emergent, repulsion-driven clustering. We draw quantitative connections between our results on a model system in naval engineering to entropy-driven phenomena in nanoscale self-assembly, and give a general argument that the clustering phenomena we observe should arise in many distributed systems. We identify circumstances under which generic design problems will exhibit trade-offs between clustering and uncertainty in design objectives, and we present a framework to identify and quantify trade-offs to manage clustering vulnerabilities.