Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

TL;DR

This paper explores integrating knowledge graphs with large language models to improve complex reasoning and reduce hallucinations, introducing novel methods for KG representation and fine-tuning LLMs.

Contribution

It introduces the first approach to represent knowledge graphs with programming language and fine-tune pretrained LLMs using KGs for enhanced reasoning.

Findings

Significant improvement in complex reasoning tasks

Enhanced interpretability of LLM reasoning

Grounded reasoning processes with structured data

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, they often struggle with complex reasoning tasks and are prone to hallucination. Recent research has shown promising results in leveraging knowledge graphs (KGs) to enhance LLM performance. KGs provide a structured representation of entities and their relationships, offering a rich source of information that can enhance the reasoning capabilities of LLMs. For this work, we have developed different techniques that tightly integrate KG structures and semantics into LLM representations. Our results show that we are able to significantly improve the performance of LLMs in complex reasoning scenarios, and ground the reasoning process with KGs. We are the first to represent KGs with programming language and fine-tune pretrained LLMs with KGs. This integration facilitates more accurate and interpretable reasoning processes, paving the way for more advanced reasoning capabilities of LLMs.