# Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema

## TL;DR

This paper explores using language models to improve data harmonization for Parkinson's disease by creating a new mapping schema and showing better results than traditional methods.

## Contribution

The paper introduces PASSIONATE, a novel Parkinson’s disease variable mapping schema for evaluating language model-based harmonization.

## Key findings

- Language model-based embeddings achieved over 80% accuracy in harmonizing Parkinson’s disease cohorts.
- Using a neighborhood of possible matches improved accuracy to up to 96%.
- Language models outperformed fuzzy string matching in automated cohort harmonization.

## Abstract

Data Harmonization is an important yet time-consuming process. With the recent popularity of applications using Language Models (LMs) due to their high capabilities in text understanding, we investigated whether LMs could facilitate data harmonization for clinical use cases. To evaluate this, we created PASSIONATE, a novel Parkinson’s disease (PD) variable mapping schema as a ground truth source for pairwise cohort harmonization using LLMs. Additionally, we extended our investigation using an existing Alzheimer’s disease (AD) CDM. We computed text embeddings based on two language models to perform automated cohort harmonization for both AD and PD. We additionally compared the results to a baseline method using fuzzy string matching to determine the degree to which the semantic capabilities of language models can be utilized for automated cohort harmonization. We found that mappings based on text embeddings performed significantly better than those generated by fuzzy string matching, reaching an average accuracy of over 80% for almost all tested PD cohorts. When extended to a further neighborhood of possible matches, the accuracy could be improved to up to 96%. Our results suggest that language models can be used for automated harmonization with a high accuracy that can potentially be improved in the future by applying domain-trained models.

The online version contains supplementary material available at 10.1038/s41598-025-06447-2.