Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine

TL;DR

This paper introduces MC-BEC, a comprehensive multimodal benchmark dataset with evaluation framework for assessing foundation models in emergency medicine, covering diverse prediction tasks from patient decompensation to ED revisits.

Contribution

It provides a large, multimodal clinical dataset and standardized evaluation framework to advance foundation models in emergency medicine.

Findings

Baseline model performances established for each prediction task.

Multimodal data improves prediction accuracy over unimodal approaches.

Benchmark encourages development of generalizable models for emergency care.

Abstract

We propose the Multimodal Clinical Benchmark for Emergency Care (MC-BEC), a comprehensive benchmark for evaluating foundation models in Emergency Medicine using a dataset of 100K+ continuously monitored Emergency Department visits from 2020-2022. MC-BEC focuses on clinically relevant prediction tasks at timescales from minutes to days, including predicting patient decompensation, disposition, and emergency department (ED) revisit, and includes a standardized evaluation framework with train-test splits and evaluation metrics. The multimodal dataset includes a wide range of detailed clinical data, including triage information, prior diagnoses and medications, continuously measured vital signs, electrocardiogram and photoplethysmograph waveforms, orders placed and medications administered throughout the visit, free-text reports of imaging studies, and information on ED diagnosis, disposition, and subsequent revisits. We provide performance baselines for each prediction task to enable the evaluation of multimodal, multitask models. We believe that MC-BEC will encourage researchers to develop more effective, generalizable, and accessible foundation models for multimodal clinical data.