Multi-Plane Vision Transformer for Hemorrhage Classification Using Axial and Sagittal MRI Data

TL;DR

This paper introduces a multi-plane vision transformer model that effectively classifies brain hemorrhages from MRI data with varying orientations, outperforming traditional methods and CNNs.

Contribution

The proposed MP-ViT employs dual transformer encoders with cross-attention for axial and sagittal MRI data, improving hemorrhage classification accuracy.

Findings

MP-ViT outperforms ViT by 5.5% in AUC.

MP-ViT surpasses CNN architectures by 1.8% in AUC.

Model demonstrates robustness across diverse MRI orientations.

Abstract

Identifying brain hemorrhages from magnetic resonance imaging (MRI) is a critical task for healthcare professionals. The diverse nature of MRI acquisitions with varying contrasts and orientation introduce complexity in identifying hemorrhage using neural networks. For acquisitions with varying orientations, traditional methods often involve resampling images to a fixed plane, which can lead to information loss. To address this, we propose a 3D multi-plane vision transformer (MP-ViT) for hemorrhage classification with varying orientation data. It employs two separate transformer encoders for axial and sagittal contrasts, using cross-attention to integrate information across orientations. MP-ViT also includes a modality indication vector to provide missing contrast information to the model. The effectiveness of the proposed model is demonstrated with extensive experiments on real world clinical dataset consists of 10,084 training, 1,289 validation and 1,496 test subjects. MP-ViT achieved substantial improvement in area under the curve (AUC), outperforming the vision transformer (ViT) by 5.5% and CNN-based architectures by 1.8%. These results highlight the potential of MP-ViT in improving performance for hemorrhage detection when different orientation contrasts are needed.