Integrating Machine Learning Ensembles and Large Language Models for Heart Disease Prediction Using Voting Fusion

TL;DR

This study combines machine learning ensembles and large language models through voting fusion to improve heart disease prediction accuracy, demonstrating that hybrid systems outperform individual models and enhance clinical decision support.

Contribution

It introduces a hybrid fusion approach integrating ML ensembles with LLM reasoning, achieving higher accuracy in cardiovascular disease prediction than standalone models.

Findings

ML ensembles achieved 95.78% accuracy and ROC-AUC 0.96.

LLMs alone achieved 78.9% accuracy, improving slightly with few-shot learning.

Hybrid fusion reached 96.62% accuracy, outperforming individual models.

Abstract

Cardiovascular disease is the primary cause of death globally, necessitating early identification, precise risk classification, and dependable decision-support technologies. The advent of large language models (LLMs) provides new zero-shot and few-shot reasoning capabilities, even though machine learning (ML) algorithms, especially ensemble approaches like Random Forest, XGBoost, LightGBM, and CatBoost, are excellent at modeling complex, non-linear patient data and routinely beat logistic regression. This research predicts cardiovascular disease using a merged dataset of 1,190 patient records, comparing traditional machine learning models (95.78% accuracy, ROC-AUC 0.96) with open-source large language models via OpenRouter APIs. Finally, a hybrid fusion of the ML ensemble and LLM reasoning under Gemini 2.5 Flash achieved the best results (96.62% accuracy, 0.97 AUC), showing that LLMs (78.9 % accuracy) work best when combined with ML models rather than used alone. Results show that ML ensembles achieved the highest performance (95.78% accuracy, ROC-AUC 0.96), while LLMs performed moderately in zero-shot (78.9%) and slightly better in few-shot (72.6%) settings. The proposed hybrid method enhanced the strength in uncertain situations, illustrating that ensemble ML is considered the best structured tabular prediction case, but it can be integrated with hybrid ML-LLM systems to provide a minor increase and open the way to more reliable clinical decision-support tools.