SELF-PERCEPT: Introspection Improves Large Language Models' Detection of Multi-Person Mental Manipulation in Conversations

TL;DR

This paper introduces SELF-PERCEPT, a two-stage prompting framework that enhances large language models' ability to detect subtle mental manipulation in complex multi-person conversations, supported by a new dataset.

Contribution

The paper presents a novel two-stage prompting method inspired by Self-Perception Theory and a new dataset for detecting manipulation in multi-turn, multi-person dialogues.

Findings

SELF-PERCEPT improves detection accuracy over baseline models.

State-of-the-art LLMs struggle with manipulation detection.

The dataset captures realistic manipulation scenarios.

Abstract

Mental manipulation is a subtle yet pervasive form of abuse in interpersonal communication, making its detection critical for safeguarding potential victims. However, due to manipulation's nuanced and context-specific nature, identifying manipulative language in complex, multi-turn, and multi-person conversations remains a significant challenge for large language models (LLMs). To address this gap, we introduce the MultiManip dataset, comprising 220 multi-turn, multi-person dialogues balanced between manipulative and non-manipulative interactions, all drawn from reality shows that mimic real-world scenarios. For manipulative interactions, it includes 11 distinct manipulations depicting real-life scenarios. We conduct extensive evaluations of state-of-the-art LLMs, such as GPT-4o and Llama-3.1-8B, employing various prompting strategies. Despite their capabilities, these models often struggle to detect manipulation effectively. To overcome this limitation, we propose SELF-PERCEPT, a novel, two-stage prompting framework inspired by Self-Perception Theory, demonstrating strong performance in detecting multi-person, multi-turn mental manipulation. Our code and data are publicly available at https://github.com/danushkhanna/self-percept .