GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding

TL;DR

This paper presents GLaM, a fine-tuning approach that aligns large language models with domain knowledge graphs, enhancing their ability to perform structured reasoning and reduce hallucinations by converting graphs into labeled text data.

Contribution

The paper introduces a novel fine-tuning framework that transforms knowledge graphs into labeled question-answer pairs, enabling LLMs to better reason over domain-specific graph structures.

Findings

Improved reasoning over domain knowledge graphs.

Enhanced factual accuracy and reduced hallucination in LLMs.

Efficient alternative to retrieval-augmented generation methods.

Abstract

Integrating large language models (LLMs) with knowledge graphs derived from domain-specific data represents an important advancement towards more powerful and factual reasoning. As these models grow more capable, it is crucial to enable them to perform multi-step inferences over real-world knowledge graphs while minimizing hallucination. While large language models excel at conversation and text generation, their ability to reason over domain-specialized graphs of interconnected entities remains limited. For example, can we query a LLM to identify the optimal contact in a professional network for a specific goal, based on relationships and attributes in a private database? The answer is no--such capabilities lie beyond current methods. However, this question underscores a critical technical gap that must be addressed. Many high-value applications in areas such as science, security, and e-commerce rely on proprietary knowledge graphs encoding unique structures, relationships, and logical constraints. We introduce a fine-tuning framework for developing Graph-aligned LAnguage Models (GLaM) that transforms a knowledge graph into an alternate text representation with labeled question-answer pairs. We demonstrate that grounding the models in specific graph-based knowledge expands the models' capacity for structure-based reasoning. Our methodology leverages the large-language model's generative capabilities to create the dataset and proposes an efficient alternate to retrieval-augmented generation styled methods.