BRACTIVE: A Brain Activation Approach to Human Visual Brain Learning

TL;DR

BRACTIVE is a transformer-based framework that aligns human visual brain activity with neural network features, identifying brain regions across multiple subjects and improving machine learning performance.

Contribution

It introduces a novel multi-subject brain region identification method that leverages fMRI data to enhance neural network training and understanding of visual processing.

Findings

Effectively identifies person-specific brain regions such as face and body areas.

Aligns neural network features with human brain activity, improving benchmark performance.

Demonstrates potential for neuroscience and machine learning applications.

Abstract

The human brain is a highly efficient processing unit, and understanding how it works can inspire new algorithms and architectures in machine learning. In this work, we introduce a novel framework named Brain Activation Network (BRACTIVE), a transformer-based approach to studying the human visual brain. The primary objective of BRACTIVE is to align the visual features of subjects with their corresponding brain representations using functional Magnetic Resonance Imaging (fMRI) signals. It enables us to identify the brain's Regions of Interest (ROIs) in the subjects. Unlike previous brain research methods, which can only identify ROIs for one subject at a time and are limited by the number of subjects, BRACTIVE automatically extends this identification to multiple subjects and ROIs. Our experiments demonstrate that BRACTIVE effectively identifies person-specific regions of interest, such as face and body-selective areas, aligning with neuroscience findings and indicating potential applicability to various object categories. More importantly, we found that leveraging human visual brain activity to guide deep neural networks enhances performance across various benchmarks. It encourages the potential of BRACTIVE in both neuroscience and machine intelligence studies.