How hate spreads online and why it returns: Re-entrant phases driven by collective behavior

TL;DR

This paper models online hate content spread using a coalescence-fragmentation framework, revealing re-entrant phase transitions that inform how moderation policies can unintentionally backfire.

Contribution

It introduces a novel two-species model incorporating empirical features of hate content spread and derives analytic solutions revealing complex phase behavior.

Findings

System-wide hate spread exhibits re-entrant threshold phases.

Analytic formulae identify how phase boundaries can be manipulated.

Policies reducing hate communities can initially succeed but may backfire.

Abstract

The 2025 Bondi Beach mass-shooting was perpetrated by individuals inspired by ISIS (Islamic State) propaganda that increasingly featured anti-Semitic hate content following the October 2023 start of the Israel-Palestine war. Similar stories hold for other types of hate attacks, e.g. against Muslims on May 18, 2026. There is an urgent need to get ahead of future threats by understanding how and when a newly created piece of hate content will spread system-wide online. We present a two-species coalescence-fragmentation model with Susceptible-Infected-Recovered dynamics that incorporates the following published empirical features: (1) New pieces of hate content tend to be generated and promoted by a subset of in-built communities on less regulated platforms. (2) These `hate' communities create links (hyperlinks) with each other and with non-hate communities across all platforms to form dynamically evolving clusters (i.e. coalescence) across which new hate content can then spread. (3) These clusters can get broken up by moderator shutdowns (i.e. fragmentation). We present numerical solutions and derive two levels of approximate mean-field theory: Effective Medium Theory (EMT) and Beyond Effective Medium Theory (BEMT). Both numerical and analytic solutions reveal that system-wide spreading is governed by re-entrant threshold phases: as the fraction of hate communities varies, the system can transition from spreading to no-spreading and back to spreading. The derived analytic formulae give explicit insight into how these phase boundaries might be manipulated to prevent system-wide spreading. More broadly, the re-entrant phase behavior warns that policies which steadily reduce the number of hate communities can initially succeed but then backfire if pushed further, suggesting that blanket requirements for platforms to simply do `more' are over-simplistic.