Reinforcement learning in social networks

TL;DR

This paper introduces a reinforcement learning model for network formation where individuals adapt their communication probabilities based on past interactions, leading to stable, star-shaped communication structures.

Contribution

It generalizes previous models by incorporating reinforcement learning into network formation, analyzing convergence and stability of communication patterns.

Findings

Expected communication rate increases and converges over time.

Stable configurations have star-shaped connected components.

Occupation measures converge to stable equilibria with specific graph structures.

Abstract

We propose a model of network formation based on reinforcement learning, which can be seen as a generalization as the one proposed by Skyrms for signaling games. On a discrete graph, whose vertices represent individuals, at any time step each of them picks one of its neighbors with a probability proportional to their past number of communications; independently, Nature chooses, with an independent identical distribution in time, which ones are allowed to communicate. Communications occur when any two neighbors mutually pick each other and are both allowed by Nature to communicate. Our results generalize the ones obtained by Hu, Skyrms and Tarr\`es (2011). We prove that, up to an error term, the expected rate of communications increases in average, and thus a.s. converges. If we define the limit graph as the non-oriented subgraph on which edges are pairs of vertices communicating with a positive asymptotic rate, then, for stable configurations, within which every vertex is connected to at least another one, the connected components of this limit graph are star-shaped and satisfy a certain balance condition. Conversely, given any stable equilibrium $q$ whose associated graph satisfies that property, the occupation measure converges with positive probability to a stable equilibrium in a neighborhood of $q$ with the same limit graph.