Decomposing 3D Neuroimaging into 2+1D Processing for Schizophrenia Recognition

TL;DR

This paper introduces a 2+1D deep learning framework that decomposes 3D neuroimaging data into 2D slices for effective schizophrenia recognition, leveraging pre-trained 2D CNNs and multi-view fusion.

Contribution

The study proposes a novel 2+1D processing method that utilizes pre-trained 2D CNNs for 3D neuroimaging analysis, improving recognition accuracy over existing methods.

Findings

Outperforms handcrafted feature-based machine learning methods.

Achieves better cross-validation results than 3D CNNs trained from scratch.

Successfully replicates results on independent datasets.

Abstract

Deep learning has been successfully applied to recognizing both natural images and medical images. However, there remains a gap in recognizing 3D neuroimaging data, especially for psychiatric diseases such as schizophrenia and depression that have no visible alteration in specific slices. In this study, we propose to process the 3D data by a 2+1D framework so that we can exploit the powerful deep 2D Convolutional Neural Network (CNN) networks pre-trained on the huge ImageNet dataset for 3D neuroimaging recognition. Specifically, 3D volumes of Magnetic Resonance Imaging (MRI) metrics (grey matter, white matter, and cerebrospinal fluid) are decomposed to 2D slices according to neighboring voxel positions and inputted to 2D CNN models pre-trained on the ImageNet to extract feature maps from three views (axial, coronal, and sagittal). Global pooling is applied to remove redundant information as the activation patterns are sparsely distributed over feature maps. Channel-wise and slice-wise convolutions are proposed to aggregate the contextual information in the third view dimension unprocessed by the 2D CNN model. Multi-metric and multi-view information are fused for final prediction. Our approach outperforms handcrafted feature-based machine learning, deep feature approach with a support vector machine (SVM) classifier and 3D CNN models trained from scratch with better cross-validation results on publicly available Northwestern University Schizophrenia Dataset and the results are replicated on another independent dataset.