AI-assisted Protocol Information Extraction For Improved Accuracy and Efficiency in Clinical Trial Workflows

TL;DR

This paper evaluates an AI system using generative LLMs with Retrieval-Augmented Generation for automated clinical trial protocol information extraction, demonstrating improved accuracy and operational efficiency over standalone LLMs.

Contribution

It introduces a RAG-based AI approach that significantly enhances extraction accuracy and workflow speed in clinical trial protocol management.

Findings

RAG process achieves 89.0% extraction accuracy, outperforming 62.6% of standalone LLMs.

AI-assisted workflows are completed 40% faster and rated as less cognitively demanding.

Users strongly prefer AI-assisted tasks, supporting integration into clinical workflows.

Abstract

Increasing clinical trial protocol complexity, amendments, and challenges around knowledge management create significant burden for trial teams. Structuring protocol content into standard formats has the potential to improve efficiency, support documentation quality, and strengthen compliance. We evaluate an Artificial Intelligence (AI) system using generative LLMs with Retrieval-Augmented Generation (RAG) for automated clinical trial protocol information extraction. We compare the extraction accuracy of our clinical-trial-specific RAG process against that of publicly available (standalone) LLMs. We also assess the operational impact of AI-assistance on simulated extraction Clinical Research Coordinator (CRC) workflows. Our RAG process shows higher extraction accuracy (89.0%) than standalone LLMs with fine-tuned prompts (62.6%) against expert-supported reference annotations. In simulated extraction workflows, AI-assisted tasks are completed 40% faster, are rated as less cognitively demanding and are strongly preferred by users. While expert oversight remains essential, this suggests that AI-assisted extraction can enable protocol intelligence at scale, motivating the integration of similar methodologies into real-world clinical workflows to further validate its impact on feasibility, study start-up, and post-activation monitoring.