Leveraging Foundation Language Models (FLMs) for Automated Cohort Extraction from Large EHR Databases

TL;DR

This paper introduces a method using foundation language models to automate and improve the accuracy of extracting cohorts from multiple large electronic health record databases, reducing manual effort.

Contribution

The study presents a novel approach leveraging FLMs for automated column matching and query generation in multi-dataset cohort extraction, demonstrating high accuracy on MIMIC-III and eICU.

Findings

Achieves 92% top-three accuracy in column matching

Maintains accuracy as database size increases

Automates cohort extraction process effectively

Abstract

A crucial step in cohort studies is to extract the required cohort from one or more study datasets. This step is time-consuming, especially when a researcher is presented with a dataset that they have not previously worked with. When the cohort has to be extracted from multiple datasets, cohort extraction can be extremely laborious. In this study, we present an approach for partially automating cohort extraction from multiple electronic health record (EHR) databases. We formulate the guided multi-dataset cohort extraction problem in which selection criteria are first converted into queries, translating them from natural language text to language that maps to database entities. Then, using FLMs, columns of interest identified from the queries are automatically matched between the study databases. Finally, the generated queries are run across all databases to extract the study cohort. We propose and evaluate an algorithm for automating column matching on two large, popular and publicly-accessible EHR databases -- MIMIC-III and eICU. Our approach achieves a high top-three accuracy of $92\%$, correctly matching $12$ out of the $13$ columns of interest, when using a small, pre-trained general purpose language model. Furthermore, this accuracy is maintained even as the search space (i.e., size of the database) increases.