Tempera: Spatial Transformer Feature Pyramid Network for Cardiac MRI Segmentation

TL;DR

Tempera is a novel multi-input/output neural network architecture that improves simultaneous segmentation of the right ventricle in cardiac MRI by leveraging multi-scale and multi-view features with geometric transformations.

Contribution

It introduces a hybrid 2D/3D spatial transformer feature pyramid network with multi-scale inputs and a geometric target transformer for RV segmentation in cardiac MRI.

Findings

Achieves Dice scores of 0.836 (SA) and 0.798 (LA)

Reduces Hausdorff distances to 26.31 mm (SA) and 31.19 mm (LA)

Enables potential integration into clinical workflows

Abstract

Assessing the structure and function of the right ventricle (RV) is important in the diagnosis of several cardiac pathologies. However, it remains more challenging to segment the RV than the left ventricle (LV). In this paper, we focus on segmenting the RV in both short (SA) and long-axis (LA) cardiac MR images simultaneously. For this task, we propose a new multi-input/output architecture, hybrid 2D/3D geometric spatial TransformEr Multi-Pass fEature pyRAmid (Tempera). Our feature pyramid extends current designs by allowing not only a multi-scale feature output but multi-scale SA and LA input images as well. Tempera transfers learned features between SA and LA images via layer weight sharing and incorporates a geometric target transformer to map the predicted SA segmentation to LA space. Our model achieves an average Dice score of 0.836 and 0.798 for the SA and LA, respectively, and 26.31 mm and 31.19 mm Hausdorff distances. This opens up the potential for the incorporation of RV segmentation models into clinical workflows.