HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge

TL;DR

HyperG introduces a hypergraph-based framework that enhances LLMs' understanding of structured knowledge by encoding complex relationships and augmenting sparse data, improving performance on downstream tasks.

Contribution

The paper presents HyperG, a novel hypergraph-enhanced generation framework that better captures structural relationships in structured data for LLMs, addressing limitations of previous serialization and operation-based methods.

Findings

HyperG outperforms existing methods on downstream structured knowledge tasks.

HyperG effectively encodes complex structural relationships in data.

HyperG demonstrates strong generalization across different tasks.

Abstract

Given that substantial amounts of domain-specific knowledge are stored in structured formats, such as web data organized through HTML, Large Language Models (LLMs) are expected to fully comprehend this structured information to broaden their applications in various real-world downstream tasks. Current approaches for applying LLMs to structured data fall into two main categories: serialization-based and operation-based methods. Both approaches, whether relying on serialization or using SQL-like operations as an intermediary, encounter difficulties in fully capturing structural relationships and effectively handling sparse data. To address these unique characteristics of structured data, we propose HyperG, a hypergraph-based generation framework aimed at enhancing LLMs' ability to process structured knowledge. Specifically, HyperG first augment sparse data with contextual information, leveraging the generative power of LLMs, and incorporate a prompt-attentive hypergraph learning (PHL) network to encode both the augmented information and the intricate structural relationships within the data. To validate the effectiveness and generalization of HyperG, we conduct extensive experiments across two different downstream tasks requiring structured knowledge.