Automated Generation of Custom MedDRA Queries Using SafeTerm Medical Map

TL;DR

This paper introduces SafeTerm, an AI system that automatically generates relevant MedDRA queries for drug safety analysis by embedding medical terms in a vector space and ranking them based on similarity, achieving high recall and precision.

Contribution

The novel SafeTerm system uses vector space embedding and statistical clustering to automate MedDRA query generation, improving efficiency and accuracy in drug safety signal detection.

Findings

High recall (>95%) at moderate thresholds

Optimal threshold (~0.70 - 0.75) yields recall ~50% and precision ~33%

SafeTerm provides a viable supplementary method for automated query generation

Abstract

In pre-market drug safety review, grouping related adverse event terms into standardised MedDRA queries or the FDA Office of New Drugs Custom Medical Queries (OCMQs) is critical for signal detection. We present a novel quantitative artificial intelligence system that understands and processes medical terminology and automatically retrieves relevant MedDRA Preferred Terms (PTs) for a given input query, ranking them by a relevance score using multi-criteria statistical methods. The system (SafeTerm) embeds medical query terms and MedDRA PTs in a multidimensional vector space, then applies cosine similarity and extreme-value clustering to generate a ranked list of PTs. Validation was conducted against the FDA OCMQ v3.0 (104 queries), restricted to valid MedDRA PTs. Precision, recall and F1 were computed across similarity-thresholds. High recall (>95%) is achieved at moderate thresholds. Higher thresholds improve precision (up to 86%). The optimal threshold (~0.70 - 0.75) yielded recall ~50% and precision ~33%. Narrow-term PT subsets performed similarly but required slightly higher similarity thresholds. The SafeTerm AI-driven system provides a viable supplementary method for automated MedDRA query generation. A similarity threshold of ~0.60 is recommended initially, with increased thresholds for refined term selection.