Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG

TL;DR

Aligning neural network models with human EEG responses to real-world images can modestly improve their robustness against adversarial attacks, with consistent effects observed across different model initializations and architectures.

Contribution

This study demonstrates that training models to predict human EEG responses to natural images can enhance adversarial robustness, a novel approach compared to prior invasive brain data methods.

Findings

EEG prediction accuracy correlates with robustness gains.

Effects are consistent across different initializations and architectures.

Strongest EEG contribution from parieto-occipital electrodes.

Abstract

In contrast to human vision, artificial neural networks (ANNs) remain relatively susceptible to adversarial attacks. To address this vulnerability, efforts have been made to transfer inductive bias from human brains to ANNs, often by training the ANN representations to match their biological counterparts. Previous works relied on brain data acquired in rodents or primates using invasive techniques, from specific regions of the brain, under non-natural conditions (anesthetized animals), and with stimulus datasets lacking diversity and naturalness. In this work, we explored whether aligning model representations to human EEG responses to a rich set of real-world images increases robustness to ANNs. Specifically, we trained ResNet50-backbone models on a dual task of classification and EEG prediction; and evaluated their EEG prediction accuracy and robustness to adversarial attacks. We observed significant correlation between the networks' EEG prediction accuracy, often highest around 100 ms post stimulus onset, and their gains in adversarial robustness. Although effect size was limited, effects were consistent across different random initializations and robust for architectural variants. We further teased apart the data from individual EEG channels and observed strongest contribution from electrodes in the parieto-occipital regions. The demonstrated utility of human EEG for such tasks opens up avenues for future efforts that scale to larger datasets under diverse stimuli conditions with the promise of stronger effects.