Frustratingly Hard Evidence Retrieval for QA Over Books
Xiangyang Mou, Mo Yu, Bingsheng Yao, Chenghao Yang, Xiaoxiao Guo,, Saloni Potdar, Hui Su

TL;DR
This paper explores question answering over narrative books, highlighting the challenges of evidence retrieval and achieving state-of-the-art results on the NarrativeQA benchmark, while emphasizing the need for improved retrieval methods.
Contribution
It formulates BookQA as an open-domain QA task, applies state-of-the-art methods, and analyzes evidence retrieval difficulties specific to narrative books.
Findings
Achieved state-of-the-art on NarrativeQA benchmark
Evidence retrieval remains a significant challenge in BookQA
Highlights need for novel evidence retrieval solutions
Abstract
A lot of progress has been made to improve question answering (QA) in recent years, but the special problem of QA over narrative book stories has not been explored in-depth. We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We further investigate how state-of-the-art open-domain QA approaches can help BookQA. Besides achieving state-of-the-art on the NarrativeQA benchmark, our study also reveals the difficulty of evidence retrieval in books with a wealth of experiments and analysis - which necessitates future effort on novel solutions for evidence retrieval in BookQA.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Expert finding and Q&A systems · Natural Language Processing Techniques
