Scalable Bayesian modeling, monitoring and analysis of dynamic network flow data

TL;DR

This paper introduces scalable Bayesian models for real-time analysis and monitoring of dynamic network flow data, enabling interpretable insights and adaptive inference in streaming count data scenarios.

Contribution

It develops flexible state-space models and structured gravity models for streaming count data, with a Bayesian monitoring framework for real-time network analysis.

Findings

Effective real-time traffic flow analysis demonstrated on web data

Models provide interpretable insights into network dynamics

Bayesian monitoring detects deviations in network flow patterns

Abstract

Traffic flow count data in networks arise in many applications, such as automobile or aviation transportation, certain directed social network contexts, and Internet studies. Using an example of Internet browser traffic flow through site-segments of an international news website, we present Bayesian analyses of two linked classes of models which, in tandem, allow fast, scalable and interpretable Bayesian inference. We first develop flexible state-space models for streaming count data, able to adaptively characterize and quantify network dynamics efficiently in real-time. We then use these models as emulators of more structured, time-varying gravity models that allow formal dissection of network dynamics. This yields interpretable inferences on traffic flow characteristics, and on dynamics in interactions among network nodes. Bayesian monitoring theory defines a strategy for sequential model assessment and adaptation in cases when network flow data deviates from model-based predictions. Exploratory and sequential monitoring analyses of evolving traffic on a network of web site-segments in e-commerce demonstrate the utility of this coupled Bayesian emulation approach to analysis of streaming network count data.