PCoKG: Personality-aware Commonsense Reasoning with Debate

TL;DR

This paper introduces PCoKG, a large personality-aware commonsense knowledge graph constructed using LLMs and debate mechanisms, to improve personalized dialogue systems by incorporating personality traits into reasoning.

Contribution

The paper presents a novel dataset, PCoKG, that integrates personality traits into commonsense reasoning using LLMs and a debate-based refinement process, advancing personalized AI systems.

Findings

PCoKG contains over 520,000 quadruples for personality-aware reasoning.

Fine-tuning with PCoKG improves dialogue consistency and personalization.

Model performance correlates with the size of the underlying language models.

Abstract

Most commonsense reasoning models overlook the influence of personality traits, limiting their effectiveness in personalized systems such as dialogue generation. To address this limitation, we introduce the Personality-aware Commonsense Knowledge Graph (PCoKG), a structured dataset comprising 521,316 quadruples. We begin by employing three evaluators to score and filter events from the ATOMIC dataset, selecting those that are likely to elicit diverse reasoning patterns across different personality types. For knowledge graph construction, we leverage the role-playing capabilities of large language models (LLMs) to perform reasoning tasks. To enhance the quality of the generated knowledge, we incorporate a debate mechanism consisting of a proponent, an opponent, and a judge, which iteratively refines the outputs through feedback loops. We evaluate the dataset from multiple perspectives and conduct fine-tuning and ablation experiments using multiple LLM backbones to assess PCoKG's robustness and the effectiveness of its construction pipeline. Our LoRA-based fine-tuning results indicate a positive correlation between model performance and the parameter scale of the base models. Finally, we apply PCoKG to persona-based dialogue generation, where it demonstrates improved consistency between generated responses and reference outputs. This work bridges the gap between commonsense reasoning and individual cognitive differences, enabling the development of more personalized and context-aware AI systems.