Towards Logically Consistent Language Models via Probabilistic Reasoning

TL;DR

This paper introduces a probabilistic reasoning-based training objective for large language models to improve their logical consistency and factual reliability without extensive fine-tuning or external tools.

Contribution

It proposes a novel training approach that enhances LLMs' logical consistency by integrating probabilistic reasoning with external knowledge, enabling better generalization.

Findings

LLMs trained with the proposed method are more logically consistent.

The approach improves the model's ability to extrapolate to unseen facts.

Enhanced factual reliability compared to baseline models.

Abstract

Large language models (LLMs) are a promising venue for natural language understanding and generation tasks. However, current LLMs are far from reliable: they are prone to generate non-factual information and, more crucially, to contradict themselves when prompted to reason about beliefs of the world. These problems are currently addressed with large scale fine-tuning or by delegating consistent reasoning to external tools. In this work, we strive for a middle ground and introduce a training objective based on principled probabilistic reasoning that teaches a LLM to be consistent with external knowledge in the form of a set of facts and rules. Fine-tuning with our loss on a limited set of facts enables our LLMs to be more logically consistent than previous baselines and allows them to extrapolate to unseen but semantically similar factual knowledge more systematically.