KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration

TL;DR

KG-o1 is a novel approach that integrates knowledge graphs with large language models to improve multi-hop reasoning and question answering, demonstrating superior performance on multiple datasets.

Contribution

This paper introduces KG-o1, a four-stage method combining knowledge graphs with LLMs to enhance multi-hop reasoning capabilities in question answering tasks.

Findings

KG-o1 outperforms existing models on multiple datasets

Knowledge graph integration improves reasoning accuracy

Self-improving corpus enhances model training

Abstract

Large Language Models (LLMs) face challenges in knowledge-intensive reasoning tasks like classic multi-hop question and answering, which involves reasoning across multiple facts. This difficulty arises because the chain of thoughts (CoTs) generated by LLMs in such tasks often deviate from real or a priori reasoning paths. In contrast, knowledge graphs (KGs) explicitly represent the logical connections between facts through entities and relationships. This reflects a significant gap. Meanwhile, large reasoning models (LRMs), such as o1, have demonstrated that long-step reasoning significantly enhances the performance of LLMs. Building on these insights, we propose KG-o1, a four-stage approach that integrates KGs to enhance the multi-hop reasoning abilities of LLMs. We first filter out initial entities and generate complex subgraphs. Secondly, we construct logical paths for subgraphs and then use knowledge graphs to build a dataset with a complex and extended brainstorming process, which trains LLMs to imitate long-term reasoning. Finally, we employ rejection sampling to generate a self-improving corpus for direct preference optimization (DPO), further refining the LLMs reasoning abilities. We conducted experiments on two simple and two complex datasets. The results show that KG-o1 models exhibit superior performance across all tasks compared to existing LRMs.