A Bi-Pyramid Multimodal Fusion Method for the Diagnosis of Bipolar Disorders

TL;DR

This paper introduces a novel multimodal fusion model combining sMRI and fMRI data with pyramid feature extraction for improved bipolar disorder diagnosis, achieving state-of-the-art accuracy.

Contribution

A new bi-pyramid multimodal fusion approach that effectively combines sMRI and fMRI data for enhanced bipolar disorder diagnosis accuracy.

Findings

Outperforms existing methods in balanced accuracy (0.657 to 0.732)

Achieves state-of-the-art diagnosis performance

Demonstrates the effectiveness of multimodal data fusion

Abstract

Previous research on the diagnosis of Bipolar disorder has mainly focused on resting-state functional magnetic resonance imaging. However, their accuracy can not meet the requirements of clinical diagnosis. Efficient multimodal fusion strategies have great potential for applications in multimodal data and can further improve the performance of medical diagnosis models. In this work, we utilize both sMRI and fMRI data and propose a novel multimodal diagnosis model for bipolar disorder. The proposed Patch Pyramid Feature Extraction Module extracts sMRI features, and the spatio-temporal pyramid structure extracts the fMRI features. Finally, they are fused by a fusion module to output diagnosis results with a classifier. Extensive experiments show that our proposed method outperforms others in balanced accuracy from 0.657 to 0.732 on the OpenfMRI dataset, and achieves the state of the art.