GOLFer: Smaller LM-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval

TL;DR

GOLFer is a novel approach that uses smaller open-source language models with a hallucination filter and content combiner to improve query expansion in information retrieval, reducing reliance on large, costly LLMs.

Contribution

GOLFer introduces a two-module system that filters hallucinated content and effectively combines generated documents with queries, enabling high-quality retrieval with smaller models.

Findings

GOLFer outperforms other small-LM methods across multiple datasets.

It maintains competitive performance against large LLM-based methods.

The approach reduces computational costs and enhances accessibility.

Abstract

Large language models (LLMs)-based query expansion for information retrieval augments queries with generated hypothetical documents with LLMs. However, its performance relies heavily on the scale of the language models (LMs), necessitating larger, more advanced LLMs. This approach is costly, computationally intensive, and often has limited accessibility. To address these limitations, we introduce GOLFer - Smaller LMs-Generated Documents Hallucination Filter & Combiner - a novel method leveraging smaller open-source LMs for query expansion. GOLFer comprises two modules: a hallucination filter and a documents combiner. The former detects and removes non-factual and inconsistent sentences in generated documents, a common issue with smaller LMs, while the latter combines the filtered content with the query using a weight vector to balance their influence. We evaluate GOLFer alongside dominant LLM-based query expansion methods on three web search and ten low-resource datasets. Experimental results demonstrate that GOLFer consistently outperforms other methods using smaller LMs, and maintains competitive performance against methods using large-size LLMs, demonstrating its effectiveness.