TomoGRAF: A Robust and Generalizable Reconstruction Network for Single-View Computed Tomography

TL;DR

TomoGRAF is a novel neural network framework that accurately reconstructs 3D CT volumes from ultra-sparse X-ray views by incorporating physics-based modeling, outperforming existing methods and enabling applications with minimal projections.

Contribution

It introduces TomoGRAF, a physics-informed neural network that captures X-ray transport physics for ultra-sparse view CT reconstruction, demonstrating superior performance over prior approaches.

Findings

Significantly outperforms state-of-the-art methods in ultra-sparse view CT reconstruction.

Successfully reconstructs high-quality 3D volumes from one or two X-ray views.

Provides a generalizable solution for clinical applications with limited projection data.

Abstract

Computed tomography (CT) provides high spatial resolution visualization of 3D structures for scientific and clinical applications. Traditional analytical/iterative CT reconstruction algorithms require hundreds of angular data samplings, a condition that may not be met in practice due to physical and mechanical limitations. Sparse view CT reconstruction has been proposed using constrained optimization and machine learning methods with varying success, less so for ultra-sparse view CT reconstruction with one to two views. Neural radiance field (NeRF) is a powerful tool for reconstructing and rendering 3D natural scenes from sparse views, but its direct application to 3D medical image reconstruction has been minimally successful due to the differences between optical and X-ray photon transportation. Here, we develop a novel TomoGRAF framework incorporating the unique X-ray transportation physics to reconstruct high-quality 3D volumes using ultra-sparse projections without prior. TomoGRAF captures the CT imaging geometry, simulates the X-ray casting and tracing process, and penalizes the difference between simulated and ground truth CT sub-volume during training. We evaluated the performance of TomoGRAF on an unseen dataset of distinct imaging characteristics from the training data and demonstrated a vast leap in performance compared with state-of-the-art deep learning and NeRF methods. TomoGRAF provides the first generalizable solution for image-guided radiotherapy and interventional radiology applications, where only one or a few X-ray views are available, but 3D volumetric information is desired.