Large Language Model Enhanced Knowledge Representation Learning: A Survey

TL;DR

This survey reviews how Large Language Models enhance Knowledge Representation Learning by integrating textual data, addressing KG sparsity, and improving downstream task performance through various encoder and decoder-based methods.

Contribution

It provides a comprehensive overview of LLM-enhanced KRL methods, categorizing approaches and highlighting future research directions.

Findings

LLM-enhanced KRL improves knowledge graph modeling.

Encoder-based methods leverage contextual information effectively.

Decoder-based methods utilize large corpora for knowledge enhancement.

Abstract

Knowledge Representation Learning (KRL) is crucial for enabling applications of symbolic knowledge from Knowledge Graphs (KGs) to downstream tasks by projecting knowledge facts into vector spaces. Despite their effectiveness in modeling KG structural information, KRL methods are suffering from the sparseness of KGs. The rise of Large Language Models (LLMs) built on the Transformer architecture presents promising opportunities for enhancing KRL by incorporating textual information to address information sparsity in KGs. LLM-enhanced KRL methods, including three key approaches, encoder-based methods that leverage detailed contextual information, encoder-decoder-based methods that utilize a unified Seq2Seq model for comprehensive encoding and decoding, and decoder-based methods that utilize extensive knowledge from large corpora, have significantly advanced the effectiveness and generalization of KRL in addressing a wide range of downstream tasks. This work provides a broad overview of downstream tasks while simultaneously identifying emerging research directions in these evolving domains.