A Network-based Multimodal Data Fusion Approach for Characterizing Dynamic Multimodal Physiological Patterns

TL;DR

This paper introduces a novel network-based multimodal data fusion method that models physiological interactions in the human body during emotional states, using complex network analysis and benchmark datasets.

Contribution

It presents a new approach combining joint recurrence plots and temporal network metrics for multimodal physiological data fusion and analysis.

Findings

Effective modeling of physiological interactions during emotional states.

Improved understanding of complex biological subsystem dynamics.

Validation on benchmark dataset demonstrates model's utility.

Abstract

Characterizing the dynamic interactive patterns of complex systems helps gain in-depth understanding of how components interrelate with each other while performing certain functions as a whole. In this study, we present a novel multimodal data fusion approach to construct a complex network, which models the interactions of biological subsystems in the human body under emotional states through physiological responses. Joint recurrence plot and temporal network metrics are employed to integrate the multimodal information at the signal level. A benchmark public dataset of is used for evaluating our model.