IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration

TL;DR

This paper introduces IGCN, a novel image-to-graph convolutional network that performs deformable registration of 3D organ meshes from single 2D images, aiding clinical applications like radiotherapy and surgery.

Contribution

The proposed framework enables simultaneous training of 2D-to-displacement and mesh feature-to-3D displacement transformations, extending deformable registration to multiple abdominal organs.

Findings

Achieved accurate 2D/3D registration for liver, stomach, duodenum, kidney, and pancreatic cancer.

Predicted respiratory motion and deformation with clinically acceptable accuracy.

Demonstrated shape prediction benefits from multi-organ relationship modeling.

Abstract

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.