Multi-Sensory Cognitive Computing for Learning Population-level Brain Connectivity

TL;DR

This paper introduces mCOCO, a novel multi-sensory reservoir computing framework for learning interpretable, efficient, and cognitively meaningful population-level brain connectivity templates from BOLD signals, outperforming existing graph neural network methods.

Contribution

mCOCO is the first framework to integrate multi-sensory inputs with reservoir computing for brain connectivity modeling, capturing cognitive traits and dynamics efficiently.

Findings

Outperforms GNN-based templates in centeredness and discriminativeness

Enhances interpretability and cognitive modeling of brain connectivity

Efficiently captures multi-sensory memory and processing dynamics

Abstract

The generation of connectional brain templates (CBTs) has recently garnered significant attention for its potential to identify unique connectivity patterns shared across individuals. However, existing methods for CBT learning such as conventional machine learning and graph neural networks (GNNs) are hindered by several limitations. These include: (i) poor interpretability due to their black-box nature, (ii) high computational cost, and (iii) an exclusive focus on structure and topology, overlooking the cognitive capacity of the generated CBT. To address these challenges, we introduce mCOCO (multi-sensory COgnitive COmputing), a novel framework that leverages Reservoir Computing (RC) to learn population-level functional CBT from BOLD (Blood-Oxygen-level-Dependent) signals. RC's dynamic system properties allow for tracking state changes over time, enhancing interpretability and enabling the modeling of brain-like dynamics, as demonstrated in prior literature. By integrating multi-sensory inputs (e.g., text, audio, and visual data), mCOCO captures not only structure and topology but also how brain regions process information and adapt to cognitive tasks such as sensory processing, all in a computationally efficient manner. Our mCOCO framework consists of two phases: (1) mapping BOLD signals into the reservoir to derive individual functional connectomes, which are then aggregated into a group-level CBT - an approach, to the best of our knowledge, not previously explored in functional connectivity studies - and (2) incorporating multi-sensory inputs through a cognitive reservoir, endowing the CBT with cognitive traits. Extensive evaluations show that our mCOCO-based template significantly outperforms GNN-based CBT in terms of centeredness, discriminativeness, topological soundness, and multi-sensory memory retention. Our source code is available at https://github.com/basiralab/mCOCO.