Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning

TL;DR

This paper uses deep reinforcement learning to discover and optimize social learning strategies in cooperative environments, revealing spontaneous emergence of complex behaviors and outperforming traditional methods.

Contribution

It introduces a framework where agents learn social strategies autonomously, demonstrating emergent behaviors and superior performance across diverse social settings.

Findings

Agents spontaneously learn social behaviors like copying and focusing on high-performing neighbors.

The learned strategies outperform traditional baseline social learning strategies.

The framework adapts well to changing environments and real social networks.

Abstract

How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.