Benchmarking Clinical Decision Support Search
Vincent Nguyen, Sarvnaz Karimi, Sara Falamaki, Cecile Paris

TL;DR
This paper presents a benchmarking framework for clinical decision support search systems, enabling comparison of different retrieval techniques and facilitating reproducible research in evidence-based medicine.
Contribution
It introduces a stable platform for benchmarking clinical search methods, allowing systematic comparison and statistical testing of various document and query processing techniques.
Findings
Different methods show varied effectiveness
Benchmarking enables reproducibility and comparison
Statistical analysis supports hypothesis testing
Abstract
Finding relevant literature underpins the practice of evidence-based medicine. From 2014 to 2016, TREC conducted a clinical decision support track, wherein participants were tasked with finding articles relevant to clinical questions posed by physicians. In total, 87 teams have participated over the past three years, generating 395 runs. During this period, each team has trialled a variety of methods. While there was significant overlap in the methods employed by different teams, the results were varied. Due to the diversity of the platforms used, the results arising from the different techniques are not directly comparable, reducing the ability to build on previous work. By using a stable platform, we have been able to compare different document and query processing techniques, allowing us to experiment with different search parameters. We have used our system to reproduce leading…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBiomedical Text Mining and Ontologies · Data Quality and Management · Health Sciences Research and Education
