Reliable Automated Triage in Spanish Clinical Notes: A Hybrid Framework for Risk-Aware HIV Suspicion Identification

TL;DR

This paper introduces a risk-aware hybrid framework for HIV suspicion detection in Spanish clinical notes, improving reliability by explicitly managing uncertainties and avoiding overconfident predictions.

Contribution

It presents a novel hybrid selective classification method that decouples types of uncertainty, enhancing safe medical triage in NLP applications.

Findings

Standard uncertainty metrics are insufficient for safe triage.

The proposed framework isolates a trustworthy operational domain.

Empirical results show improved reliability over baseline classifiers.

Abstract

Standard clinical Natural Language Processing (NLP) benchmarks often yield inflated metrics by forcing deterministic classification on ambiguous instances, thereby obscuring the clinical risks of overconfident predictions. To bridge this gap, we propose a risk-aware hybrid selective classification framework, evaluated on early Human Immunodeficiency Virus suspicion identification in Spanish clinical notes. Our dual-verification approach explicitly decouples aleatoric uncertainty through Mondrian conformal prediction and epistemic uncertainty using a Multi-Centroid Mahalanobis Distance veto. Empirical evaluations reveal that standard uncertainty metrics and baseline classifiers are structurally insufficient for safe medical triage, suffering severe coverage collapse when forced to operate under strict reliability constraints. In contrast, by demanding that clinical narratives pass both probabilistic and geometric safeguards, the proposed framework successfully isolates a highly trustworthy operational domain.