Image To Tree with Recursive Prompting

TL;DR

This paper introduces a recursive prompting method combining UNet and Transformer architectures to predict tree connectivity in complex 3D anatomical images, overcoming limitations of traditional minimal path approaches in overlapping projections.

Contribution

The work presents a novel recursive optimization approach with image-based prompting, improving tree structure prediction in challenging medical imaging scenarios.

Findings

Outperforms shortest-path baseline on synthetic datasets

Effective in predicting complex tree structures with overlapping branches

Demonstrates the potential of recursive prompting in medical image analysis

Abstract

Extracting complex structures from grid-based data is a common key step in automated medical image analysis. The conventional solution to recovering tree-structured geometries typically involves computing the minimal cost path through intermediate representations derived from segmentation masks. However, this methodology has significant limitations in the context of projective imaging of tree-structured 3D anatomical data such as coronary arteries, since there are often overlapping branches in the 2D projection. In this work, we propose a novel approach to predicting tree connectivity structure which reformulates the task as an optimization problem over individual steps of a recursive process. We design and train a two-stage model which leverages the UNet and Transformer architectures and introduces an image-based prompting technique. Our proposed method achieves compelling results on a pair of synthetic datasets, and outperforms a shortest-path baseline.