MINDECHO: Role-Playing Language Agents for Key Opinion Leaders

TL;DR

MINDECHO is a framework that creates and assesses Key Opinion Leader role-playing language agents by leveraging data from internet videos and GPT-4, enabling realistic and domain-specific interactions.

Contribution

It introduces a novel framework for developing and evaluating KOL role-playing language agents using internet video data and GPT-4 synthesis.

Findings

Effective in generating KOL-specific conversations

Validated through extensive experiments

Addresses both general and fan-centric evaluation metrics

Abstract

Large language models~(LLMs) have demonstrated impressive performance in various applications, among which role-playing language agents (RPLAs) have engaged a broad user base. Now, there is a growing demand for RPLAs that represent Key Opinion Leaders (KOLs), \ie, Internet celebrities who shape the trends and opinions in their domains. However, research in this line remains underexplored. In this paper, we hence introduce MINDECHO, a comprehensive framework for the development and evaluation of KOL RPLAs. MINDECHO collects KOL data from Internet video transcripts in various professional fields, and synthesizes their conversations leveraging GPT-4. Then, the conversations and the transcripts are used for individualized model training and inference-time retrieval, respectively. Our evaluation covers both general dimensions (\ie, knowledge and tones) and fan-centric dimensions for KOLs. Extensive experiments validate the effectiveness of MINDECHO in developing and evaluating KOL RPLAs.