Composable Strategy Framework with Integrated Video-Text based Large Language Models for Heart Failure Assessment

TL;DR

This paper introduces a multi-modal, composable AI framework that integrates video, text, and clinical data to improve heart failure assessment and treatment planning, outperforming single-modal models in prognosis accuracy.

Contribution

The paper presents a novel framework that simulates doctor-patient interactions and combines diverse data sources for holistic heart failure evaluation.

Findings

Multi-modal approach outperforms single-modal AI in prognosis accuracy.

Framework enables comprehensive evaluation of pathological indicators.

Simulates clinical consultation process for better treatment optimization.

Abstract

Heart failure is one of the leading causes of death worldwide, with millons of deaths each year, according to data from the World Health Organization (WHO) and other public health agencies. While significant progress has been made in the field of heart failure, leading to improved survival rates and improvement of ejection fraction, there remains substantial unmet needs, due to the complexity and multifactorial characteristics. Therefore, we propose a composable strategy framework for assessment and treatment optimization in heart failure. This framework simulates the doctor-patient consultation process and leverages multi-modal algorithms to analyze a range of data, including video, physical examination, text results as well as medical history. By integrating these various data sources, our framework offers a more holistic evaluation and optimized treatment plan for patients. Our results demonstrate that this multi-modal approach outperforms single-modal artificial intelligence (AI) algorithms in terms of accuracy in heart failure (HF) prognosis prediction. Through this method, we can further evaluate the impact of various pathological indicators on HF prognosis,providing a more comprehensive evaluation.