EviSearch: A Human in the Loop System for Extracting and Auditing Clinical Evidence for Systematic Reviews

TL;DR

EviSearch is a multi-agent system that automates extraction of clinical evidence from PDFs, ensuring provenance and enabling human verification, to improve systematic review workflows.

Contribution

The paper introduces EviSearch, a novel multi-agent pipeline for high-precision, ontology-aligned evidence extraction with provenance tracking from clinical trial PDFs.

Findings

Significantly improves extraction accuracy over baselines.

Provides comprehensive attribution coverage for evidence.

Facilitates iterative model improvement through logged decisions.

Abstract

We present EviSearch, a multi-agent extraction system that automates the creation of ontology-aligned clinical evidence tables directly from native trial PDFs while guaranteeing per-cell provenance for audit and human verification. EviSearch pairs a PDF-query agent (which preserves rendered layout and figures) with a retrieval-guided search agent and a reconciliation module that forces page-level verification when agents disagree. The pipeline is designed for high-precision extraction across multimodal evidence sources (text, tables, figures) and for generating reviewer-actionable provenance that clinicians can inspect and correct. On a clinician-curated benchmark of oncology trial papers, EviSearch substantially improves extraction accuracy relative to strong parsed-text baselines while providing comprehensive attribution coverage. By logging reconciler decisions and reviewer edits, the system produces structured preference and supervision signals that bootstrap iterative model improvement. EviSearch is intended to accelerate living systematic review workflows, reduce manual curation burden, and provide a safe, auditable path for integrating LLM-based extraction into evidence synthesis pipelines.