Encoding the Local Connectivity Patterns of fMRI for Cognitive State Classification

TL;DR

This paper introduces a novel framework using Fisher Vectors, VLAD, and BoW to encode local brain connectivity patterns from fMRI data, improving cognitive state classification accuracy.

Contribution

It presents a new method for encoding local connectivity patterns with Fisher Vectors and compares it to existing encoding methods, enhancing cognitive state classification.

Findings

FV encoding outperforms VLAD and BoW in classification accuracy

Significant Gaussians in GMM influence classification performance

Proposed visualization method for brain connectivity codewords

Abstract

In this work, we propose a novel framework to encode the local connectivity patterns of brain, using Fisher Vectors (FV), Vector of Locally Aggregated Descriptors (VLAD) and Bag-of-Words (BoW) methods. We first obtain local descriptors, called Mesh Arc Descriptors (MADs) from fMRI data, by forming local meshes around anatomical regions, and estimating their relationship within a neighborhood. Then, we extract a dictionary of relationships, called \textit{brain connectivity dictionary} by fitting a generative Gaussian mixture model (GMM) to a set of MADs, and selecting the codewords at the mean of each component of the mixture. Codewords represent the connectivity patterns among anatomical regions. We also encode MADs by VLAD and BoW methods using the k-Means clustering. We classify the cognitive states of Human Connectome Project (HCP) task fMRI dataset, where we train support vector machines (SVM) by the encoded MADs. Results demonstrate that, FV encoding of MADs can be successfully employed for classification of cognitive tasks, and outperform the VLAD and BoW representations. Moreover, we identify the significant Gaussians in mixture models by computing energy of their corresponding FV parts, and analyze their effect on classification accuracy. Finally, we suggest a new method to visualize the codewords of brain connectivity dictionary.