Options-Aware Dense Retrieval for Multiple-Choice query Answering
Manish Singh, Manish Shrivastava

TL;DR
This paper introduces Options Aware Dense Retrieval (OADR), a novel fine-tuning method for dense retrieval models that improves evidence identification in long-context multiple-choice question answering tasks.
Contribution
OADR leverages query-options embeddings to better mimic oracle query embeddings, enhancing evidence retrieval for multiple-choice QA.
Findings
OADR outperforms existing baselines on QuALITY dataset.
Enhanced evidence retrieval leads to higher accuracy in multiple-choice QA.
Fine-tuning with query-options embeddings improves retrieval robustness.
Abstract
Long-context multiple-choice question answering tasks require robust reasoning over extensive text sources. Since most of the pre-trained transformer models are restricted to processing only a few hundred words at a time, successful completion of such tasks often relies on the identification of evidence spans, such as sentences, that provide supporting evidence for selecting the correct answer. Prior research in this domain has predominantly utilized pre-trained dense retrieval models, given the absence of supervision to fine-tune the retrieval process. This paper proposes a novel method called Options Aware Dense Retrieval (OADR) to address these challenges. ORDA uses an innovative approach to fine-tuning retrieval by leveraging query-options embeddings, which aim to mimic the embeddings of the oracle query (i.e., the query paired with the correct answer) for enhanced identification of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Management and Algorithms · Advanced Image and Video Retrieval Techniques · Algorithms and Data Compression
MethodsAttentive Walk-Aggregating Graph Neural Network
