Domain-specific Question Answering with Hybrid Search
Dewang Sultania, Zhaoyu Lu, Twisha Naik, Franck Dernoncourt, David, Seunghyun Yoon, Sanat Sharma, Trung Bui, Ashok Gupta, Tushar Vatsa, Suhas, Suresha, Ishita Verma, Vibha Belavadi, Cheng Chen, Michael Friedrich

TL;DR
This paper presents a hybrid retrieval approach combining dense and sparse methods with weighted relevance signals, significantly improving domain-specific question answering accuracy in enterprise contexts.
Contribution
It introduces a novel hybrid retrieval system that combines dense and sparse search techniques with tunable relevance weights for enhanced domain-specific QA.
Findings
Hybrid method outperforms single-retriever systems.
Improved accuracy in domain-specific question answering.
Effective integration of multiple relevance signals.
Abstract
Domain specific question answering is an evolving field that requires specialized solutions to address unique challenges. In this paper, we show that a hybrid approach combining a fine-tuned dense retriever with keyword based sparse search methods significantly enhances performance. Our system leverages a linear combination of relevance signals, including cosine similarity from dense retrieval, BM25 scores, and URL host matching, each with tunable boost parameters. Experimental results indicate that this hybrid method outperforms our single-retriever system, achieving improved accuracy while maintaining robust contextual grounding. These findings suggest that integrating multiple retrieval methodologies with weighted scoring effectively addresses the complexities of domain specific question answering in enterprise settings.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Service-Oriented Architecture and Web Services
MethodsUmbrella Reinforcement Learning
