Transforming Questions and Documents for Semantically Aligned Retrieval-Augmented Generation

TL;DR

This paper presents a novel retrieval-augmented generation framework for multihop question answering that decomposes questions with LLMs and uses question-generated document embeddings for improved retrieval accuracy.

Contribution

It introduces a new approach combining LLM-based question decomposition and question-generated document embeddings for enhanced multihop QA performance.

Findings

Improved RAG performance on multihop datasets

Effective question decomposition reduces ambiguity

Question-generated embeddings outperform raw document embeddings

Abstract

We introduce a novel retrieval-augmented generation (RAG) framework tailored for multihop question answering. First, our system uses large language model (LLM) to decompose complex multihop questions into a sequence of single-hop subquestions that guide document retrieval. This decomposition mitigates the ambiguity inherent in multi-hop queries by clearly targeting distinct knowledge facets. Second, instead of embedding raw or chunked documents directly, we generate answerable questions from each document chunk using Qwen3-8B, embed these generated questions, and retrieve relevant chunks via question-question embedding similarity. During inference, the retrieved chunks are then fed along with the original question into the RAG pipeline. We evaluate on three multihop question datasets (MuSiQue, 2WikiMultiHopQa, HotpotQA) from LongBench. Our method improves RAG performacne compared to baseline systems. Our contributions highlight the benefits of using answerable-question embeddings for RAG, and the effectiveness of LLM-based query decomposition for multihop scenarios.