SymbolicThought: Integrating Language Models and Symbolic Reasoning for Consistent and Interpretable Human Relationship Understanding

TL;DR

SymbolicThought is a human-in-the-loop framework that combines language models and symbolic reasoning to improve the accuracy, consistency, and interpretability of character relationship understanding in narratives.

Contribution

It introduces a novel interactive system integrating LLMs with symbolic reasoning to construct and refine character relationship graphs with logical constraints.

Findings

Improves annotation accuracy and consistency

Reduces time cost in relationship annotation

Supports explainable social analysis

Abstract

Understanding character relationships is essential for interpreting complex narratives and conducting socially grounded AI research. However, manual annotation is time-consuming and low in coverage, while large language models (LLMs) often produce hallucinated or logically inconsistent outputs. We present SymbolicThought, a human-in-the-loop framework that combines LLM-based extraction with symbolic reasoning. The system constructs editable character relationship graphs, refines them using seven types of logical constraints, and enables real-time validation and conflict resolution through an interactive interface. To support logical supervision and explainable social analysis, we release a dataset of 160 interpersonal relationships with corresponding logical structures. Experiments show that SymbolicThought improves annotation accuracy and consistency while significantly reducing time cost, offering a practical tool for narrative understanding, explainable AI, and LLM evaluation.