Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains

TL;DR

This paper introduces a conformal prediction framework that provides reliable confidence guarantees for medical entity extraction using large language models across different clinical domains, addressing calibration issues.

Contribution

It demonstrates domain-specific conformal calibration for LLM-based medical extraction, ensuring safe deployment with coverage guarantees despite calibration heterogeneity.

Findings

Models are underconfident on FDA labels, overconfident on radiology reports.

Conformal prediction achieves ≥90% coverage with low rejection rates.

Calibration depends on document structure, category, and model architecture.

Abstract

Large Language Models (LLMs) are increasingly used for medical entity extraction, yet their confidence scores are often miscalibrated, limiting safe deployment in clinical settings. We present a conformal prediction framework that provides finite-sample coverage guarantees for LLM-based extraction across two clinical domains. First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1, verified via FactScore-based atomic statement evaluation (97.7\% accuracy over 128,906 entities). Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick, evaluated against physician annotations (entity F1: 0.81 to 0.84). Our central finding is that miscalibration direction reverses across domains: on well-structured FDA labels, models are underconfident, requiring modest conformal thresholds ($\tau \approx 0.06$), while on free-text radiology reports, models are overconfident, demanding strict thresholds ($\tau$ up to 0.99). Despite this heterogeneity, conformal prediction achieves target coverage ($\geq 90\%$) in both settings with manageable rejection rates (9--13\%). These results demonstrate that calibration is not a global model property but depends on document structure, extraction category, and model architecture, motivating domain-specific conformal calibration for safe clinical deployment.