Deformation equivariant cross-modality image synthesis with paired non-aligned training data

TL;DR

This paper introduces a novel method for cross-modality image synthesis that effectively handles paired but misaligned training data by incorporating deformation equivariance, enabling more robust clinical applications.

Contribution

It proposes a new deformation equivariance loss and joint training framework with registration networks for improved synthesis with misaligned data.

Findings

Effective handling of non-aligned paired data

Improved synthesis quality on real-world datasets

Facilitates clinical application of cross-modality synthesis

Abstract

Cross-modality image synthesis is an active research topic with multiple medical clinically relevant applications. Recently, methods allowing training with paired but misaligned data have started to emerge. However, no robust and well-performing methods applicable to a wide range of real world data sets exist. In this work, we propose a generic solution to the problem of cross-modality image synthesis with paired but non-aligned data by introducing new deformation equivariance encouraging loss functions. The method consists of joint training of an image synthesis network together with separate registration networks and allows adversarial training conditioned on the input even with misaligned data. The work lowers the bar for new clinical applications by allowing effortless training of cross-modality image synthesis networks for more difficult data sets.