QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domain

TL;DR

The paper introduces QCG-Rerank, a novel reranking method that enhances retrieval quality for tourism-related queries in LLMs by expanding queries and constructing a chunks graph for better answer generation.

Contribution

It proposes a new reranking model that combines query expansion and chunks graph construction to improve retrieval relevance in tourism domain LLM applications.

Findings

QCG-Rerank outperforms existing methods on multiple datasets.

Enhanced query expansion improves retrieval accuracy.

Iterative transition probability computation effectively ranks chunks.

Abstract

Retrieval-Augmented Generation (RAG) mitigates the issue of hallucination in Large Language Models (LLMs) by integrating information retrieval techniques. However, in the tourism domain, since the query is usually brief and the content in the database is diverse, existing RAG may contain a significant amount of irrelevant or contradictory information contents after retrieval. To address this challenge, we propose the QCG-Rerank model. This model first performs an initial retrieval to obtain candidate chunks and then enhances semantics by extracting critical information to expand the original query. Next, we utilize the expanded query and candidate chunks to calculate similarity scores as the initial transition probability and construct the chunks graph. Subsequently, We iteratively compute the transition probabilities based on an initial estimate until convergence. The chunks with the highest score are selected and input into the LLMs to generate responses. We evaluate the model on Cultour, IIRC, StrategyQA, HotpotQA, SQuAD, and MuSiQue datasets. The experimental results demonstrate the effectiveness and superiority of the QCG-Rerank method.