Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansion

TL;DR

This survey reviews current methods for expanding large language models' knowledge, emphasizing techniques like continual learning, model editing, and retrieval-based adaptation to improve their flexibility and robustness.

Contribution

It provides a comprehensive overview of state-of-the-art knowledge expansion techniques for LLMs, highlighting challenges and future opportunities.

Findings

Various knowledge integration techniques are discussed.

Challenges like knowledge consistency and scalability are identified.

The survey guides future research directions in LLM knowledge adaptation.

Abstract

Adapting large language models (LLMs) to new and diverse knowledge is essential for their lasting effectiveness in real-world applications. This survey provides an overview of state-of-the-art methods for expanding the knowledge of LLMs, focusing on integrating various knowledge types, including factual information, domain expertise, language proficiency, and user preferences. We explore techniques, such as continual learning, model editing, and retrieval-based explicit adaptation, while discussing challenges like knowledge consistency and scalability. Designed as a guide for researchers and practitioners, this survey sheds light on opportunities for advancing LLMs as adaptable and robust knowledge systems.