Using higher-order Markov models to reveal flow-based communities in networks

TL;DR

This paper enhances network community detection by employing second-order Markov models that incorporate pathway memory, revealing temporal communities in dynamic systems beyond traditional first-order models.

Contribution

It introduces a second-order Markov framework for community detection, extending standard algorithms to capture temporal correlations and pathway memory in networks.

Findings

Second-order models improve community detection accuracy.

Temporal correlations reveal dynamic community structures.

Relation established between second-order models and non-backtracking matrices.

Abstract

Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the prevailing paradigm to model such a network-based dynamics, for instance in the form of random walks or other types of diffusions. Despite the success of this modelling perspective for numerous applications, it represents an over-simplification of several real-world systems. Importantly, simple Markov models lack memory in their dynamics, an assumption often not realistic in practice. Here, we explore possibilities to enrich the system description by means of second-order Markov models, exploiting empirical pathway information. We focus on the problem of community detection and show that standard network algorithms can be generalized in order to extract novel temporal information about the system under investigation. We also apply our methodology to temporal networks, where we can uncover communities shaped by the temporal correlations in the system. Finally, we discuss relations of the framework of second order Markov processes and the recently proposed formalism of using non-backtracking matrices for community detection.