Axial-Centric Cross-Plane Attention for 3D Medical Image Classification

TL;DR

This paper introduces an axial-centric cross-plane attention model for 3D medical image classification, reflecting clinical workflows by emphasizing the axial plane and integrating multi-plane information for improved accuracy.

Contribution

The proposed architecture models asymmetric dependencies between anatomical planes, incorporating a pretrained medical vision model and novel transformer modules for enhanced 3D image analysis.

Findings

Outperforms existing models on MedMNIST3D datasets in accuracy and AUC.

Ablation studies confirm the effectiveness of axial-centric attention and directional cross-plane fusion.

Aligning model design with clinical workflows improves robustness and data efficiency.

Abstract

Clinicians commonly interpret three-dimensional (3D) medical images, such as computed tomography (CT) scans, using multiple anatomical planes rather than as a single volumetric representation. In this multi-planar approach, the axial plane typically serves as the primary acquisition and diagnostic reference, while the coronal and sagittal planes provide complementary spatial information to increase diagnostic confidence. However, many existing 3D deep learning methods either process volumetric data holistically or assign equal importance to all planes, failing to reflect the axial-centric clinical interpretation workflow. To address this gap, we propose an axial-centric cross-plane attention architecture for 3D medical image classification that captures the inherent asymmetric dependencies between different anatomical planes. Our architecture incorporates MedDINOv3, a medical vision foundation model pretrained via self-supervised learning on large-scale axial CT images, as a frozen feature extractor for the axial, coronal, and sagittal planes. RICA blocks and intra-plane transformer encoders capture plane-specific positional and contextual information within each anatomical plane, while axial-centric cross-plane transformer encoders condition axial features on complementary information from auxiliary planes. Experimental results on six datasets from the MedMNIST3D benchmark demonstrate that the proposed architecture consistently outperforms existing 3D and multi-plane models in terms of accuracy and AUC. Ablation studies further confirm the importance of axial-centric query-key-value allocation and directional cross-plane fusion. These results highlight the importance of aligning architectural design with clinical interpretation workflows for robust and data-efficient 3D medical image analysis.