Hierarchical Heuristic Learning towards Effcient Norm Emergence

TL;DR

This paper introduces a Hierarchically Heuristic Learning Strategy (HHLS) that accelerates the emergence of social norms in multiagent systems through hierarchical information sharing and heuristic updates, outperforming previous methods.

Contribution

The paper presents a novel hierarchical learning framework that improves the efficiency and applicability of norm emergence in large multiagent environments.

Findings

HHLS supports faster norm emergence compared to previous methods.

Hierarchical structure enhances coordination in diverse multiagent scenarios.

Component analysis shows the importance of hierarchical and non-hierarchical factors.

Abstract

Social norms serve as an important mechanism to regulate the behaviors of agents and to facilitate coordination among them in multiagent systems. One important research question is how a norm can rapidly emerge through repeated local interaction within an agent society under different environments when their coordination space becomes large. To address this problem, we propose a Hierarchically Heuristic Learning Strategy (HHLS) under the hierarchical social learning framework, in which subordinate agents report their information to their supervisors, while supervisors can generate instructions (rules and suggestions) based on the information collected from their subordinates. Subordinate agents heuristically update their strategies based on both their own experience and the instructions from their supervisors. Extensive experiment evaluations show that HHLS can support the emergence of desirable social norms more efficiently and is applicable in a much wider range of multiagent interaction scenarios compared with previous work. We also investigate the effectiveness of HHLS by separating out the different components of the HHLS and evaluating the relative importance of those components. The influence of key related factors (e.g., hierarchical factors, non-hierarchical factors, fixed-strategy agents) are investigated as well.