AutoHealth: An Uncertainty-Aware Multi-Agent System for Autonomous Health Data Modeling

TL;DR

AutoHealth introduces an uncertainty-aware multi-agent system that autonomously models diverse health data, enhancing predictive accuracy and reliability for healthcare decision-making.

Contribution

It presents a novel multi-agent framework that autonomously handles health data modeling with integrated uncertainty estimation, addressing generalization and adaptability issues.

Findings

AutoHealth outperforms state-of-the-art baselines by 29.2% in prediction performance.

AutoHealth improves uncertainty estimation accuracy by 50.2%.

The system successfully completes 17 diverse real-world health data tasks.

Abstract

LLM-based agents have demonstrated strong potential for autonomous machine learning, yet their applicability to health data remains limited. Existing systems often struggle to generalize across heterogeneous health data modalities, rely heavily on predefined solution templates with insufficient adaptation to task-specific objectives, and largely overlook uncertainty estimation, which is essential for reliable decision-making in healthcare. To address these challenges, we propose \textit{AutoHealth}, a novel uncertainty-aware multi-agent system that autonomously models health data and assesses model reliability. \textit{AutoHealth} employs closed-loop coordination among five specialized agents to perform data exploration, task-conditioned model construction, training, and optimization, while jointly prioritizing predictive performance and uncertainty quantification. Beyond producing ready-to-use models, the system generates comprehensive reports to support trustworthy interpretation and risk-aware decision-making. To rigorously evaluate its effectiveness, we curate a challenging real-world benchmark comprising 17 tasks across diverse data modalities and learning settings. \textit{AutoHealth} completes all tasks and outperforms state-of-the-art baselines by 29.2\% in prediction performance and 50.2\% in uncertainty estimation.