LGM: Enhancing Large Language Models with Conceptual Meta-Relations and Iterative Retrieval

TL;DR

The paper introduces the LGM, a model that enhances large language models by extracting and validating conceptual meta-relations and dynamically retrieving related information, leading to improved interpretation and response accuracy without context length limitations.

Contribution

The paper presents the LGM framework that extracts meta-relations from natural language, employs a reflection mechanism, and uses iterative retrieval to improve LLM understanding beyond traditional RAG methods.

Findings

LGM outperforms existing RAG baselines on standard benchmarks.

The model effectively extracts and validates meta-relations from natural language.

LGM enables processing texts of any length without truncation.

Abstract

Large language models (LLMs) exhibit strong semantic understanding, yet struggle when user instructions involve ambiguous or conceptually misaligned terms. We propose the Language Graph Model (LGM) to enhance conceptual clarity by extracting meta-relations-inheritance, alias, and composition-from natural language. The model further employs a reflection mechanism to validate these meta-relations. Leveraging a Concept Iterative Retrieval Algorithm, these relations and related descriptions are dynamically supplied to the LLM, improving its ability to interpret concepts and generate accurate responses. Unlike conventional Retrieval-Augmented Generation (RAG) approaches that rely on extended context windows, our method enables large language models to process texts of any length without the need for truncation. Experiments on standard benchmarks demonstrate that the LGM consistently outperforms existing RAG baselines.