Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

TL;DR

This paper introduces MERA, a novel model that enhances large language models for clinical diagnosis prediction by combining hierarchical contrastive learning and concept memorization, achieving state-of-the-art results on MIMIC datasets.

Contribution

MERA is the first to integrate hierarchical contrastive learning and concept memorization in LLMs for clinical diagnosis prediction, addressing data scarcity and large decision spaces.

Findings

MERA outperforms existing models on MIMIC-III and IV datasets.

It significantly improves diagnosis prediction accuracy.

The approach elevates generative LLM capabilities in clinical tasks.

Abstract

Clinical diagnosis prediction models, when provided with a patient's medical history, aim to detect potential diseases early, facilitating timely intervention and improving prognostic outcomes. However, the inherent scarcity of patient data and large disease candidate space often pose challenges in developing satisfactory models for this intricate task. The exploration of leveraging Large Language Models (LLMs) for encapsulating clinical decision processes has been limited. We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes. Experimental results on MIMIC-III and IV datasets show that MERA achieves the state-of-the-art diagnosis prediction performance and dramatically elevates the diagnosis prediction capabilities of generative LMs.