SOTOPIA-$\Omega$: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents

TL;DR

This paper introduces SOTOPIA-Ω, a framework that dynamically injects social strategies into language agents to improve their social capabilities and evaluation, surpassing expert benchmarks like GPT-4.

Contribution

The paper presents a novel dynamic strategy injection method and new social instruction following metrics to enhance and evaluate social skills in language agents.

Findings

Models trained with the proposed corpus outperform GPT-4 in social goal achievement.

Dynamic construction helps break prolonged deadlocks in social dialogues.

Enhanced models show improved social instruction following performance.

Abstract

Despite the abundance of prior social strategies possessed by humans, there remains a paucity of research dedicated to their transfer and integration into social agents. Our proposed SOTOPIA-$\Omega$ framework aims to address and bridge this gap, with a particular focus on enhancing the social capabilities of language agents. This framework dynamically injects multi-step reasoning strategies inspired by negotiation theory and two simple direct strategies into expert agents, thereby automating the construction of a high-quality social dialogue training corpus. Additionally, we introduce the concept of Social Instruction Following (S-IF) and propose two new S-IF evaluation metrics that complement social capability. We demonstrate that several 7B models trained on high-quality corpus not only significantly surpass the expert agent (GPT-4) in achieving social goals but also enhance S-IF performance. Analysis and variant experiments validate the advantages of dynamic construction, which can especially break the agent's prolonged deadlock.