Multilevel compression of random walks on networks reveals hierarchical organization in large integrated systems

TL;DR

The paper introduces the hierarchical map equation, an information-theoretic method to uncover multilevel hierarchical structures in large networks by compressing random walk descriptions, revealing organizational patterns across various systems.

Contribution

It presents a novel multilevel community detection algorithm based on network compression, enabling the identification of hierarchical structures in large complex networks.

Findings

Uncovered hierarchical organization in air traffic networks, revealing countries and continents.

Revealed scientific disciplines and subfields in communication networks.

Identified shallow hierarchies in neural networks and complex multilevel structures in road networks.

Abstract

To comprehend the hierarchical organization of large integrated systems, we introduce the hierarchical map equation, which reveals multilevel structures in networks. In this information-theoretic approach, we exploit the duality between compression and pattern detection; by compressing a description of a random walker as a proxy for real flow on a network, we find regularities in the network that induce this system-wide flow. Finding the shortest multilevel description of the random walker therefore gives us the best hierarchical clustering of the network, the optimal number of levels and modular partition at each level, with respect to the dynamics on the network. With a novel search algorithm, we extract and illustrate the rich multilevel organization of several large social and biological networks. For example, from the global air traffic network we uncover countries and continents, and from the pattern of scientific communication we reveal more than 100 scientific fields organized in four major disciplines: life sciences, physical sciences, ecology and earth sciences, and social sciences. In general, we find shallow hierarchical structures in globally interconnected systems, such as neural networks, and rich multilevel organizations in systems with highly separated regions, such as road networks.