CAF-I: A Collaborative Multi-Agent Framework for Enhanced Irony Detection with Large Language Models

TL;DR

CAF-I introduces a multi-agent LLM framework that improves irony detection by simulating human-like analysis, achieving state-of-the-art zero-shot performance and enhancing interpretability.

Contribution

The paper presents CAF-I, a novel multi-agent system that addresses limitations of existing LLM methods in irony detection through collaborative analysis and optimization.

Findings

Achieves an average Macro-F1 of 76.31 in zero-shot irony detection.

Outperforms previous baselines with a 4.98 increase in Macro-F1.

Demonstrates improved interpretability and multi-perspective analysis.

Abstract

Large language model (LLM) have become mainstream methods in the field of sarcasm detection. However, existing LLM methods face challenges in irony detection, including: 1. single-perspective limitations, 2. insufficient comprehensive understanding, and 3. lack of interpretability. This paper introduces the Collaborative Agent Framework for Irony (CAF-I), an LLM-driven multi-agent system designed to overcome these issues. CAF-I employs specialized agents for Context, Semantics, and Rhetoric, which perform multidimensional analysis and engage in interactive collaborative optimization. A Decision Agent then consolidates these perspectives, with a Refinement Evaluator Agent providing conditional feedback for optimization. Experiments on benchmark datasets establish CAF-I's state-of-the-art zero-shot performance. Achieving SOTA on the vast majority of metrics, CAF-I reaches an average Macro-F1 of 76.31, a 4.98 absolute improvement over the strongest prior baseline. This success is attained by its effective simulation of human-like multi-perspective analysis, enhancing detection accuracy and interpretability.