Boosting Large Language Models for Mental Manipulation Detection via Data Augmentation and Distillation

TL;DR

This paper introduces MentalMAD, a comprehensive framework combining data augmentation, teacher supervision, and distillation to improve large language models in detecting covert mental manipulation on social media.

Contribution

It presents a novel multi-component approach and a new dataset for enhancing LLMs in mental manipulation detection, addressing annotation and data scarcity challenges.

Findings

MentalMAD improves detection accuracy by 14%.

Macro-F1 score increases by 27.3%.

Weighted F1 score improves by 15.1%.

Abstract

Mental manipulation on social media poses a covert yet serious threat to individuals' psychological well-being and the integrity of online interactions. Detecting such behavior is challenging due to the difficult-to-annotate training data, its highly covert and multi-turn nature, and the lack of real-world datasets. To address these challenges, we propose MentalMAD, a framework that enhances large language models for mental manipulation detection. Our approach consists of three key components: EvoSA, an annotation-free data augmentation method that combines evolutionary operations with speech-act-aware prompting; teacher-model-generated complementary-task supervision; and Complementary-Convergent Distillation, a phase-wise strategy for transferring manipulation-specific knowledge to student models. We then constructed the ReaMent dataset, comprising 5,000 real-world-sourced dialogues. Extensive experiments show that MentalMAD improves accuracy by 14.0%, macro-F1 by 27.3%, and weighted F1 by 15.1% over the strongest baseline. The code and the dataset are publicly available at https://github.com/Yuansheng-Gao/MentalMAD.