H2G2-Net: A Hierarchical Heterogeneous Graph Generative Network Framework for Discovery of Multi-Modal Physiological Responses

TL;DR

H2G2-Net is a novel hierarchical heterogeneous graph generative network that automatically learns graph structures from multi-modal physiological data, improving cognitive state prediction accuracy without domain knowledge.

Contribution

It introduces a hierarchical, data-driven graph learning approach for multi-modal physiological signals, overcoming limitations of existing GNNs requiring pre-defined graphs.

Findings

Outperforms state-of-the-art GNNs by 5%-20% in prediction accuracy.

Effectively models hierarchical, multi-modal physiological data.

Automatically learns graph structures without domain knowledge.

Abstract

Discovering human cognitive and emotional states using multi-modal physiological signals draws attention across various research applications. Physiological responses of the human body are influenced by human cognition and commonly used to analyze cognitive states. From a network science perspective, the interactions of these heterogeneous physiological modalities in a graph structure may provide insightful information to support prediction of cognitive states. However, there is no clue to derive exact connectivity between heterogeneous modalities and there exists a hierarchical structure of sub-modalities. Existing graph neural networks are designed to learn on non-hierarchical homogeneous graphs with pre-defined graph structures; they failed to learn from hierarchical, multi-modal physiological data without a pre-defined graph structure. To this end, we propose a hierarchical heterogeneous graph generative network (H2G2-Net) that automatically learns a graph structure without domain knowledge, as well as a powerful representation on the hierarchical heterogeneous graph in an end-to-end fashion. We validate the proposed method on the CogPilot dataset that consists of multi-modal physiological signals. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art GNNs by 5%-20% in prediction accuracy.