Deep Multi-Modal Classification of Intraductal Papillary Mucinous Neoplasms (IPMN) with Canonical Correlation Analysis

TL;DR

This paper introduces a novel CNN and canonical correlation analysis-based system for automatic diagnosis and risk assessment of IPMN using multi-modal MRI, achieving significant improvements over existing methods.

Contribution

It is the first to automatically diagnose IPMN with multi-modal MRI using a CNN and CCA for feature fusion, enhancing diagnostic accuracy.

Findings

Significant improvement over other methods in IPMN classification

Effective multi-modal MRI feature fusion using CCA

No explicit sample balancing needed for class imbalance

Abstract

Pancreatic cancer has the poorest prognosis among all cancer types. Intraductal Papillary Mucinous Neoplasms (IPMNs) are radiographically identifiable precursors to pancreatic cancer; hence, early detection and precise risk assessment of IPMN are vital. In this work, we propose a Convolutional Neural Network (CNN) based computer aided diagnosis (CAD) system to perform IPMN diagnosis and risk assessment by utilizing multi-modal MRI. In our proposed approach, we use minimum and maximum intensity projections to ease the annotation variations among different slices and type of MRIs. Then, we present a CNN to obtain deep feature representation corresponding to each MRI modality (T1-weighted and T2-weighted). At the final step, we employ canonical correlation analysis (CCA) to perform a fusion operation at the feature level, leading to discriminative canonical correlation features. Extracted features are used for classification. Our results indicate significant improvements over other potential approaches to solve this important problem. The proposed approach doesn't require explicit sample balancing in cases of imbalance between positive and negative examples. To the best of our knowledge, our study is the first to automatically diagnose IPMN using multi-modal MRI.