A theory of web traffic

TL;DR

This paper models web traffic dynamics using branching processes, revealing that web pages operate near a critical state with power-law traffic distributions, explaining bursty access patterns and web-wide self-organization.

Contribution

It introduces a novel application of branching process theory to model web traffic and explains the emergence of criticality and power-law behavior in web page access statistics.

Findings

Web traffic exhibits bursty, non-homogeneous patterns.

Websites self-organize into a critical state with power-law traffic distribution.

The model explains the origin of traffic avalanches and web-wide criticality.

Abstract

We analyze access statistics of several popular webpages for a period of several years. The graphs of daily downloads are highly non-homogeneous with long periods of low activity interrupted by bursts of heavy traffic. These bursts are due to avalanches of blog entries, referring to the page. We quantitatively explain this behavior using the theory of branching processes. We extrapolate these findings to construct a model of the entire web. According to the model, the competition between webpages for viewers pushes the web into a self-organized critical state. In this regime, the most interesting webpages are in a near-critical state, with a power law distribution of traffic intensity.