Medical Video Generation for Disease Progression Simulation

TL;DR

This paper introduces a novel Medical Video Generation framework that uses large language models and diffusion techniques to create realistic, personalized simulations of disease progression across various medical imaging domains, aiding diagnosis, education, and data interpolation.

Contribution

The paper presents the first controllable MVG framework combining LLMs and diffusion models for realistic disease progression simulation in medical images.

Findings

Outperforms baseline models in generating coherent disease trajectories

Validated across chest X-ray, fundus, and skin images

Physician user studies confirm clinical plausibility and utility

Abstract

Modeling disease progression is crucial for improving the quality and efficacy of clinical diagnosis and prognosis, but it is often hindered by a lack of longitudinal medical image monitoring for individual patients. To address this challenge, we propose the first Medical Video Generation (MVG) framework that enables controlled manipulation of disease-related image and video features, allowing precise, realistic, and personalized simulations of disease progression. Our approach begins by leveraging large language models (LLMs) to recaption prompt for disease trajectory. Next, a controllable multi-round diffusion model simulates the disease progression state for each patient, creating realistic intermediate disease state sequence. Finally, a diffusion-based video transition generation model interpolates disease progression between these states. We validate our framework across three medical imaging domains: chest X-ray, fundus photography, and skin image. Our results demonstrate that MVG significantly outperforms baseline models in generating coherent and clinically plausible disease trajectories. Two user studies by veteran physicians, provide further validation and insights into the clinical utility of the generated sequences. MVG has the potential to assist healthcare providers in modeling disease trajectories, interpolating missing medical image data, and enhancing medical education through realistic, dynamic visualizations of disease progression.