Investigating the Impact of Subgraph Social Structure Preference on the Strategic Behavior of Networked Mixed-Motive Learning Agents

TL;DR

This paper explores how agents' preferences for specific social subgraph structures influence their strategic behavior in social dilemmas, revealing consistent behavioral patterns and introducing a new structural variation metric.

Contribution

It introduces SRIM to model social structure preferences, demonstrates their impact on agent strategies, and proposes the BCI metric for analyzing structural variations across environments.

Findings

Preferences over subgraph structures affect agents' reward and strategy.

Agents with different structural positions show similar behavioral shifts.

BCI metric reliably captures structural variation across environments.

Abstract

Limited work has examined the strategic behaviors of relational networked learning agents under social dilemmas, and has overlooked the intricate social dynamics of complex systems. We address the challenge with Socio-Relational Intrinsic Motivation (SRIM), which endows agents with diverse preferences over sub-graphical social structures in order to study the impact of agents' personal preferences over their sub-graphical relations on their strategic decision-making under sequential social dilemmas. Our results in the Harvest and Cleanup environments demonstrate that preferences over different subgraph structures (degree-, clique-, and critical connection-based) lead to distinct variations in agents' reward gathering and strategic behavior: individual aggressiveness in Harvest and individual contribution effort in Cleanup. Moreover, agents with different subgraphical structural positions consistently exhibit similar strategic behavioral shifts. Our proposed BCI metric captures structural variation within the population, and the relative ordering of BCI across social preferences is consistent in Harvest and Cleanup games for the same topology, suggesting the subgraphical structural impact is robust across environments. These results provide a new lens for examining agents' behavior in social dilemmas and insight for designing effective multi-agent ecosystems composed of heterogeneous social agents.