A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications

TL;DR

This systematic review analyzes intermediate fusion techniques in multimodal deep learning for biomedical applications, highlighting current methods, challenges, and future directions to improve model integration of diverse data types.

Contribution

It formalizes and categorizes intermediate fusion methods in biomedical MDL, introducing structured notation to aid understanding and application beyond the biomedical domain.

Findings

Comprehensive analysis of intermediate fusion techniques

Identification of key challenges and future research directions

Introduction of a structured notation for better understanding

Abstract

Deep learning has revolutionized biomedical research by providing sophisticated methods to handle complex, high-dimensional data. Multimodal deep learning (MDL) further enhances this capability by integrating diverse data types such as imaging, textual data, and genetic information, leading to more robust and accurate predictive models. In MDL, differently from early and late fusion methods, intermediate fusion stands out for its ability to effectively combine modality-specific features during the learning process. This systematic review aims to comprehensively analyze and formalize current intermediate fusion methods in biomedical applications. We investigate the techniques employed, the challenges faced, and potential future directions for advancing intermediate fusion methods. Additionally, we introduce a structured notation to enhance the understanding and application of these methods beyond the biomedical domain. Our findings are intended to support researchers, healthcare professionals, and the broader deep learning community in developing more sophisticated and insightful multimodal models. Through this review, we aim to provide a foundational framework for future research and practical applications in the dynamic field of MDL.