Investigating Tool-Memory Conflicts in Tool-Augmented LLMs

TL;DR

This paper identifies a new type of knowledge conflict, Tool-Memory Conflict, in tool-augmented LLMs, especially affecting STEM tasks, and shows current conflict resolution methods are ineffective.

Contribution

It introduces the concept of Tool-Memory Conflict in LLMs, analyzes its impact on performance, and evaluates existing conflict resolution techniques, revealing their limitations.

Findings

Existing LLMs suffer from Tool-Memory Conflict, especially on STEM tasks.

Current conflict resolution methods are ineffective against TMC.

Tool-Memory Conflict varies with different conditions, affecting model prioritization.

Abstract

Tool-augmented large language models (LLMs) have powered many applications. However, they are likely to suffer from knowledge conflict. In this paper, we propose a new type of knowledge conflict -- Tool-Memory Conflict (TMC), where the internal parametric knowledge contradicts with the external tool knowledge for tool-augmented LLMs. We find that existing LLMs, though powerful, suffer from TMC, especially on STEM-related tasks. We also uncover that under different conditions, tool knowledge and parametric knowledge may be prioritized differently. We then evaluate existing conflict resolving techniques, including prompting-based and RAG-based methods. Results show that none of these approaches can effectively resolve tool-memory conflicts.