Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents

TL;DR

This paper presents Agent Hospital, a simulated environment where autonomous LLM-powered medical agents evolve through treatment interactions, outperforming existing methods on medical exam benchmarks and offering broad application potential.

Contribution

Introduction of a hospital simulacrum with evolving LLM-based medical agents that improve through treatment interactions without manual data labeling.

Findings

Evolved doctor agents outperform state-of-the-art on MedQA benchmark.

Simulacrum enables large-scale agent evolution in a simulated medical environment.

Potential applications extend beyond medical AI.

Abstract

The recent rapid development of large language models (LLMs) has sparked a new wave of technological revolution in medical artificial intelligence (AI). While LLMs are designed to understand and generate text like a human, autonomous agents that utilize LLMs as their "brain" have exhibited capabilities beyond text processing such as planning, reflection, and using tools by enabling their "bodies" to interact with the environment. We introduce a simulacrum of hospital called Agent Hospital that simulates the entire process of treating illness, in which all patients, nurses, and doctors are LLM-powered autonomous agents. Within the simulacrum, doctor agents are able to evolve by treating a large number of patient agents without the need to label training data manually. After treating tens of thousands of patient agents in the simulacrum (human doctors may take several years in the real world), the evolved doctor agents outperform state-of-the-art medical agent methods on the MedQA benchmark comprising US Medical Licensing Examination (USMLE) test questions. Our methods of simulacrum construction and agent evolution have the potential in benefiting a broad range of applications beyond medical AI.