TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network

TL;DR

This paper introduces TCFNet, a Transformer-based coarse-to-fine network for precise face-bone point cloud transformation in surgical simulation, overcoming limitations of previous methods in processing large-scale data and local detail accuracy.

Contribution

The paper presents a novel end-to-end Transformer-based framework with local information aggregation for dense face-bone transformations, incorporating an auxiliary loss for organ reconstruction.

Findings

Outperforms state-of-the-art methods on datasets

Achieves high evaluation metrics and visualization quality

Effectively models local geometric structures

Abstract

Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed to view this problem as a point-to-point transformation between skeletal and facial point clouds. However, these approaches cannot process large-scale points, have limited receptive fields that lead to noisy points, and employ complex preprocessing and postprocessing operations based on registration. These shortcomings limit the performance and widespread applicability of such methods. Therefore, we propose a Transformer-based coarse-to-fine point movement network (TCFNet) to learn unique, complicated correspondences at the patch and point levels for dense face-bone point cloud transformations. This end-to-end framework adopts a Transformer-based network and a local information aggregation network (LIA-Net) in the first and second stages, respectively, which reinforce each other to generate precise point movement paths. LIA-Net can effectively compensate for the neighborhood precision loss of the Transformer-based network by modeling local geometric structures (edges, orientations and relative position features). The previous global features are employed to guide the local displacement using a gated recurrent unit. Inspired by deformable medical image registration, we propose an auxiliary loss that can utilize expert knowledge for reconstructing critical organs.Compared with the existing state-of-the-art (SOTA) methods on gathered datasets, TCFNet achieves outstanding evaluation metrics and visualization results. The code is available at https://github.com/Runshi-Zhang/TCFNet.