In-context Interference in Chat-based Large Language Models

TL;DR

This paper investigates how in-context information flow in chat-based large language models can cause interference, leading to forgetting previously learned knowledge and reduced performance, and introduces a benchmark to evaluate this issue.

Contribution

It identifies and analyzes in-context interference in LLMs and proposes a new benchmark based on the bAbI dataset to evaluate this phenomenon.

Findings

In-context information can cause models to forget prior knowledge.

Interference reduces model performance on tasks.

A new benchmark for measuring in-context interference is introduced.

Abstract

Large language models (LLMs) have had a huge impact on society due to their impressive capabilities and vast knowledge of the world. Various applications and tools have been created that allow users to interact with these models in a black-box scenario. However, one limitation of this scenario is that users cannot modify the internal knowledge of the model, and the only way to add or modify internal knowledge is by explicitly mentioning it to the model during the current interaction. This learning process is called in-context training, and it refers to training that is confined to the user's current session or context. In-context learning has significant applications, but also has limitations that are seldom studied. In this paper, we present a study that shows how the model can suffer from interference between information that continually flows in the context, causing it to forget previously learned knowledge, which can reduce the model's performance. Along with showing the problem, we propose an evaluation benchmark based on the bAbI dataset.