Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models

TL;DR

This paper investigates how Retrieval-Augmented Language Models (RALMs) are affected by imperfect retrieval scenarios, revealing their vulnerabilities and proposing new methods to evaluate and improve their robustness against unanswerable, adversarial, and conflicting information.

Contribution

The study provides the first comprehensive analysis of RALMs' robustness to imperfect retrieval, introduces a new adversarial attack method GenADV and a robustness metric RAD, and highlights critical vulnerabilities.

Findings

RALMs often fail to detect unanswerable or contradictory documents.

Adversarial attacks significantly degrade RALM performance.

Vulnerabilities increase when adversarial and unanswerable scenarios overlap.

Abstract

Retrieval Augmented Language Models (RALMs) have gained significant attention for their ability to generate accurate answer and improve efficiency. However, RALMs are inherently vulnerable to imperfect information due to their reliance on the imperfect retriever or knowledge source. We identify three common scenarios-unanswerable, adversarial, conflicting-where retrieved document sets can confuse RALM with plausible real-world examples. We present the first comprehensive investigation to assess how well RALMs detect and handle such problematic scenarios. Among these scenarios, to systematically examine adversarial robustness we propose a new adversarial attack method, Generative model-based ADVersarial attack (GenADV) and a novel metric Robustness under Additional Document (RAD). Our findings reveal that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations. Moreover, we show the addition of an adversary significantly degrades RALM's performance, with the model becoming even more vulnerable when the two scenarios overlap (adversarial+unanswerable). Our research identifies critical areas for assessing and enhancing the robustness of RALMs, laying the foundation for the development of more robust models.