EHR-Based Mobile and Web Platform for Chronic Disease Risk Prediction Using Large Language Multimodal Models

TL;DR

This paper presents a novel EHR-based platform leveraging large language multimodal models to predict chronic disease risks, integrating clinical notes and blood tests with real-time diagnostics for clinical use.

Contribution

It introduces a new AI-powered prediction system that combines multimodal data from EHRs with web and mobile interfaces, enabling real-time risk assessment for chronic diseases.

Findings

Successful integration of LLMMs with EHR data

Real-time risk prediction accessible via web and mobile

Demonstrated effectiveness in clinical scenarios

Abstract

Traditional diagnosis of chronic diseases involves in-person consultations with physicians to identify the disease. However, there is a lack of research focused on predicting and developing application systems using clinical notes and blood test values. We collected five years of Electronic Health Records (EHRs) from Taiwan's hospital database between 2017 and 2021 as an AI database. Furthermore, we developed an EHR-based chronic disease prediction platform utilizing Large Language Multimodal Models (LLMMs), successfully integrating with frontend web and mobile applications for prediction. This prediction platform can also connect to the hospital's backend database, providing physicians with real-time risk assessment diagnostics. The demonstration link can be found at https://www.youtube.com/watch?v=oqmL9DEDFgA.