END: Early Noise Dropping for Efficient and Effective Context Denoising

TL;DR

END introduces a method to improve LLM performance by early noise removal from input sequences, enhancing efficiency and accuracy without fine-tuning.

Contribution

The paper presents a novel early noise dropping technique that leverages LLMs' implicit understanding to filter noisy input chunks before token generation.

Findings

END improves LLM performance across multiple datasets.

END reduces computational overhead during inference.

The approach enhances understanding of LLM internal reasoning processes.

Abstract

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, they are often distracted by irrelevant or noisy context in input sequences that degrades output quality. This problem affects both long- and short-context scenarios, such as retrieval-augmented generation, table question-answering, and in-context learning. We reveal that LLMs can implicitly identify whether input sequences contain useful information at early layers, prior to token generation. Leveraging this insight, we introduce Early Noise Dropping (\textsc{END}), a novel approach to mitigate this issue without requiring fine-tuning the LLMs. \textsc{END} segments input sequences into chunks and employs a linear prober on the early layers of LLMs to differentiate between informative and noisy chunks. By discarding noisy chunks early in the process, \textsc{END} preserves critical information, reduces distraction, and lowers computational overhead. Extensive experiments demonstrate that \textsc{END} significantly improves both performance and efficiency across different LLMs on multiple evaluation datasets. Furthermore, by investigating LLMs' implicit understanding to the input with the prober, this work also deepens understanding of how LLMs do reasoning with contexts internally.