SEVADE: Self-Evolving Multi-Agent Analysis with Decoupled Evaluation for Hallucination-Resistant Irony Detection

TL;DR

SEVADE introduces a multi-agent framework with decoupled evaluation to improve sarcasm detection accuracy and robustness against hallucinations, leveraging specialized linguistic agents and a reasoning chain.

Contribution

The paper presents a novel self-evolving multi-agent analysis framework with decoupled evaluation, enhancing sarcasm detection by reducing hallucination effects and improving performance.

Findings

Achieves state-of-the-art accuracy and Macro-F1 scores on four benchmark datasets.

Demonstrates 6.75% average improvement in accuracy.

Shows 6.29% average improvement in Macro-F1 score.

Abstract

Sarcasm detection is a crucial yet challenging Natural Language Processing task. Existing Large Language Model methods are often limited by single-perspective analysis, static reasoning pathways, and a susceptibility to hallucination when processing complex ironic rhetoric, which impacts their accuracy and reliability. To address these challenges, we propose **SEVADE**, a novel **S**elf-**Ev**olving multi-agent **A**nalysis framework with **D**ecoupled **E**valuation for hallucination-resistant sarcasm detection. The core of our framework is a Dynamic Agentive Reasoning Engine (DARE), which utilizes a team of specialized agents grounded in linguistic theory to perform a multifaceted deconstruction of the text and generate a structured reasoning chain. Subsequently, a separate lightweight rationale adjudicator (RA) performs the final classification based solely on this reasoning chain. This decoupled architecture is designed to mitigate the risk of hallucination by separating complex reasoning from the final judgment. Extensive experiments on four benchmark datasets demonstrate that our framework achieves state-of-the-art performance, with average improvements of **6.75%** in Accuracy and **6.29%** in Macro-F1 score.