KoACD: The First Korean Adolescent Dataset for Cognitive Distortion Analysis via Role-Switching Multi-LLM Negotiation

TL;DR

This paper introduces KoACD, a large-scale Korean adolescent dataset for cognitive distortion analysis, utilizing multi-LLM negotiation and synthetic data generation to improve classification and understanding of adolescent mental health issues.

Contribution

It presents the first extensive dataset of adolescent cognitive distortions in Korean, along with a novel multi-LLM negotiation method and synthetic data generation techniques.

Findings

LLMs classify explicit distortion markers effectively

Context-dependent reasoning remains challenging for LLMs

Expert evaluations outperform LLMs in complex cases

Abstract

Cognitive distortion refers to negative thinking patterns that can lead to mental health issues like depression and anxiety in adolescents. Previous studies using natural language processing (NLP) have focused mainly on small-scale adult datasets, with limited research on adolescents. This study introduces KoACD, the first large-scale dataset of cognitive distortions in Korean adolescents, containing 108,717 instances. We applied a multi-Large Language Model (LLM) negotiation method to refine distortion classification, enabling iterative feedback and role-switching between models to reduce bias and improve label consistency. In addition, we generated synthetic data using two approaches: cognitive clarification for textual clarity and cognitive balancing for diverse distortion representation. Validation through LLMs and expert evaluations showed that while LLMs classified distortions with explicit markers, they struggled with context-dependent reasoning, where human evaluators demonstrated higher accuracy. KoACD aims to enhance future research on cognitive distortion detection. The dataset and implementation details are publicly accessible.