QAEncoder: Towards Aligned Representation Learning in Question Answering Systems

TL;DR

QAEncoder is a training-free method that improves question-answering systems by aligning query and document embeddings, reducing retrieval gaps without extra training or storage.

Contribution

It introduces QAEncoder, a novel, training-free approach that enhances embedding alignment in QA systems, addressing the query-document gap effectively.

Findings

Effective across multiple datasets and languages

No additional storage or training required

Reduces hallucination and forgetting issues

Abstract

Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses. However, the inherent gap between user queries and relevant documents hinders precise matching. We introduce QAEncoder, a training-free approach to bridge this gap. Specifically, QAEncoder estimates the expectation of potential queries in the embedding space as a robust surrogate for the document embedding, and attaches document fingerprints to effectively distinguish these embeddings. Extensive experiments across diverse datasets, languages, and embedding models confirmed QAEncoder's alignment capability, which offers a simple-yet-effective solution with zero additional index storage, retrieval latency, training costs, or catastrophic forgetting and hallucination issues. The repository is publicly available at https://github.com/IAAR-Shanghai/QAEncoder.