LLMs can be easily Confused by Instructional Distractions

TL;DR

This paper introduces DIM-Bench, a new benchmark to evaluate how large language models struggle with instructional distraction, revealing their vulnerability to confusion when input resembles instructions across various tasks.

Contribution

The paper presents DIM-Bench, a novel benchmark for assessing LLM performance under instructional distraction, highlighting the models' susceptibility to confusion in such scenarios.

Findings

LLMs often fail to follow user intent under instructional distraction

Even advanced LLMs are vulnerable to confusion caused by instructional-like inputs

DIM-Bench categorizes real-world instances of instructional distraction across multiple tasks

Abstract

Despite the fact that large language models (LLMs) show exceptional skill in instruction following tasks, this strength can turn into a vulnerability when the models are required to disregard certain instructions. Instruction-following tasks typically involve a clear task description and input text containing the target data to be processed. However, when the input itself resembles an instruction, confusion may arise, even if there is explicit prompting to distinguish between the task instruction and the input. We refer to this phenomenon as instructional distraction. In this paper, we introduce a novel benchmark, named DIM-Bench, specifically designed to assess LLMs' performance under instructional distraction. The benchmark categorizes real-world instances of instructional distraction and evaluates LLMs across four instruction tasks: rewriting, proofreading, translation, and style transfer -- alongside five input tasks: reasoning, code generation, mathematical reasoning, bias detection, and question answering. Our experimental results reveal that even the most advanced LLMs are susceptible to instructional distraction, often failing to accurately follow user intent in such cases.