iTRIALSPACE: Programmable Virtual Lesion Trials for Controlled Evaluation of Lung CT Models

TL;DR

iTRIALSPACE is a flexible framework for controlled, virtual lung CT lesion trials that enables detailed evaluation of model accuracy and robustness beyond traditional static benchmarks.

Contribution

It introduces a novel pipeline for synthesizing controlled lung CT datasets using real clinical data and lesion profiles, facilitating more precise model assessment.

Findings

Synthetic CTs maintain real-to-real FID baseline.

Performance rankings transfer with high correlation ($\rho$=0.93).

Controlled modes reveal insights like size prediction collapse and variance ratios.

Abstract

We introduce iTRIALSPACE, a programmable evaluation framework for controlled assessment of lung CT models. Standard benchmarks are static retrospective collections that entangle lesion size, lobe prevalence, anatomy, and acquisition context, making it difficult to determine what structurally drives model accuracy. iTRIALSPACE addresses this limitation by composing real clinical CTs and lesion profiles into controlled virtual lesion trials through a four-stage pipeline: multidataset nodule profiling, explicit trial specification, anatomy-aware mask insertion, and ControlNet-conditioned CT synthesis. The framework is built on a unified 54-attribute nodule-profile dataset spanning 13,140 annotated nodules from seven public CT sources and instantiated as 13 trial modes. We evaluate iTRIALSPACE in a 55,469-sample Virtual Lesion Study spanning three medical VLMs, four spatialguidance conditions, and three clinical tasks. Across all 13 modes, the synthetic substrate remains within the real-to-real FID baseline, and synthetic performance rankings transfer strongly to real clinical data ($\rho$ = 0.93, p < 10$^{-15}$). Controlled trial modes expose findings unavailable to fixed-distribution benchmarks, including shortcut-driven size prediction collapse under lobe-equalized sampling and hostto-donor variance ratios of 8.9x and 3.3x in twin-cross analysis. These results position iTRIALSPACE as an auditable evaluation infrastructure for controlled, falsifiable testing beyond static retrospective benchmarks.