Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling

TL;DR

This paper reviews methods for integrating knowledge graphs into large language models to improve their factual accuracy and reasoning abilities, and proposes a new approach called KGLLMs for further enhancement.

Contribution

It introduces the concept of KGLLMs, a novel framework that combines knowledge graphs with LLMs to improve factual reasoning capabilities.

Findings

Enhanced LLMs with KGs improve factual recall.

Knowledge graph integration boosts knowledge-grounded response quality.

Proposed KGLLMs open new research directions in factual language modeling.

Abstract

Recently, ChatGPT, a representative large language model (LLM), has gained considerable attention due to its powerful emergent abilities. Some researchers suggest that LLMs could potentially replace structured knowledge bases like knowledge graphs (KGs) and function as parameterized knowledge bases. However, while LLMs are proficient at learning probabilistic language patterns based on large corpus and engaging in conversations with humans, they, like previous smaller pre-trained language models (PLMs), still have difficulty in recalling facts while generating knowledge-grounded contents. To overcome these limitations, researchers have proposed enhancing data-driven PLMs with knowledge-based KGs to incorporate explicit factual knowledge into PLMs, thus improving their performance to generate texts requiring factual knowledge and providing more informed responses to user queries. This paper reviews the studies on enhancing PLMs with KGs, detailing existing knowledge graph enhanced pre-trained language models (KGPLMs) as well as their applications. Inspired by existing studies on KGPLM, this paper proposes to enhance LLMs with KGs by developing knowledge graph-enhanced large language models (KGLLMs). KGLLM provides a solution to enhance LLMs' factual reasoning ability, opening up new avenues for LLM research.