How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances

TL;DR

This paper reviews recent methods for keeping large language models current with evolving world knowledge without full re-training, highlighting challenges and future research directions.

Contribution

It systematically categorizes and compares recent approaches for updating LLMs with new information efficiently.

Findings

Various techniques for model updating are analyzed

Challenges in maintaining up-to-date knowledge are identified

Future research directions are proposed

Abstract

Although large language models (LLMs) are impressive in solving various tasks, they can quickly be outdated after deployment. Maintaining their up-to-date status is a pressing concern in the current era. This paper provides a comprehensive review of recent advances in aligning LLMs with the ever-changing world knowledge without re-training from scratch. We categorize research works systemically and provide in-depth comparisons and discussion. We also discuss existing challenges and highlight future directions to facilitate research in this field. We release the paper list at https://github.com/hyintell/awesome-refreshing-llms