Image To Tree with Recursive Prompting
James Batten, Matthew Sinclair, Ben Glocker, Michiel Schaap

TL;DR
This paper introduces a recursive prompting method combining UNet and Transformer architectures to predict tree connectivity in complex 3D anatomical images, overcoming limitations of traditional minimal path approaches in overlapping projections.
Contribution
The work presents a novel recursive optimization approach with image-based prompting, improving tree structure prediction in challenging medical imaging scenarios.
Findings
Outperforms shortest-path baseline on synthetic datasets
Effective in predicting complex tree structures with overlapping branches
Demonstrates the potential of recursive prompting in medical image analysis
Abstract
Extracting complex structures from grid-based data is a common key step in automated medical image analysis. The conventional solution to recovering tree-structured geometries typically involves computing the minimal cost path through intermediate representations derived from segmentation masks. However, this methodology has significant limitations in the context of projective imaging of tree-structured 3D anatomical data such as coronary arteries, since there are often overlapping branches in the 2D projection. In this work, we propose a novel approach to predicting tree connectivity structure which reformulates the task as an optimization problem over individual steps of a recursive process. We design and train a two-stage model which leverages the UNet and Transformer architectures and introduces an image-based prompting technique. Our proposed method achieves compelling results on a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Image Segmentation Techniques · Explainable Artificial Intelligence (XAI) · Advanced Neural Network Applications
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Layer Normalization · Softmax · Adam · Byte Pair Encoding · Residual Connection · Label Smoothing · Dropout
