Evaluation of YTEX and MetaMap for clinical concept recognition

TL;DR

This study compares MetaMap and YTEX for clinical concept recognition, showing MetaMap's superior precision in strict tasks and YTEX's better overall performance in relaxed tasks and UMLS CUI mapping.

Contribution

It provides a comparative evaluation of MetaMap and YTEX without modifications, highlighting their strengths in different clinical concept recognition scenarios.

Findings

MetaMap outperforms YTEX in strict task precision.

Y T E X achieves higher overall F-Score in relaxed tasks.

Y T E X has 1.3% higher accuracy in UMLS CUI mapping.

Abstract

We used MetaMap and YTEX as a basis for the construc- tion of two separate systems to participate in the 2013 ShARe/CLEF eHealth Task 1[9], the recognition of clinical concepts. No modifications were directly made to these systems, but output concepts were filtered using stop concepts, stop concept text and UMLS semantic type. Con- cept boundaries were also adjusted using a small collection of rules to increase precision on the strict task. Overall MetaMap had better per- formance than YTEX on the strict task, primarily due to a 20% perfor- mance improvement in precision. In the relaxed task YTEX had better performance in both precision and recall giving it an overall F-Score 4.6% higher than MetaMap on the test data. Our results also indicated a 1.3% higher accuracy for YTEX in UMLS CUI mapping.