Synchronization of mean-field models on the circle

TL;DR

This paper introduces a new criterion for synchronization in mean-field models on the circle, applies it to self-attention dynamics, and extends the known parameter range for synchronization.

Contribution

It proposes a general synchronization criterion based on the third derivative of the interaction function and resolves a conjecture for self-attention models.

Findings

Synchronization occurs for all β ≥ -0.16

Synchronization does not occur for β < -2/3

Extends the parameter range for synchronization in self-attention models

Abstract

This paper considers a mean-field model of $n$ interacting particles whose state space is the unit circle, a generalization of the classical Kuramoto model. Global synchronization is said to occur if after starting from almost any initial state, all particles coalesce to a common point on the circle. We propose a general synchronization criterion in terms of $L_1$-norm of the third derivative of the particle interaction function. As an application we resolve a conjecture for the so-called self-attention dynamics (stylized model of transformers), by showing synchronization for all $\beta \ge -0.16$, which significantly extends the previous bound of $0\le \beta \le 1$ from Criscitiello, Rebjock, McRae, and Boumal (2024). We also show that global synchronization does not occur when $\beta < -2/3$.