Dissociating spatial frequency reliance from adversarial robustness advantages in neurally guided deep convolutional neural networks

TL;DR

This study investigates whether the adversarial robustness of neurally aligned deep CNNs is primarily due to reliance on low spatial frequencies or the human visual channel, finding that spectral bias alone does not account for robustness.

Contribution

It demonstrates that biasing CNNs towards the human visual channel does not improve robustness, whereas low spatial frequency bias yields modest gains, highlighting the complexity of neural alignment effects.

Findings

Neural alignment increases reliance on both LSF and the human channel.

Biasing models towards the human channel does not improve robustness.

LSF bias yields modest robustness improvements.

Abstract

Deep convolutional neural networks (DCNNs) have rivaled humans on many visual tasks, yet they remain vulnerable to near-imperceptible perturbations generated by adversarial attacks. Recent work shows that aligning DCNN representations with human visual cortex activity improves adversarial robustness, but the mechanisms driving this advantage are unclear. One hypothesis suggests that neural alignment confers robustness by biasing models away from brittle high-frequency details and towards the low spatial frequencies (LSF). However, recent work shows that human object recognition critically depends on a narrow, mid-frequency "human channel". Interestingly, this band was partially preserved in prior LSF-focused studies. Here, we investigate whether a spectral bias towards the LSF or the human channel is the primary driver of the adversarial robustness observed in neurally aligned DCNNs. We first show that DCNNs aligned to higher-order regions of the human ventral visual stream systematically increase reliance on both LSF and the human channel. However, directly steering DCNNs towards these bands revealed a clear dissociation. Biasing models towards the human channel, either alone or together with LSF, does not improve robustness and even impairs it. LSF bias produced some robustness gains, but such improvements are modest despite inducing much larger shifts in spatial-frequency reliance than neurally aligned models. Spatial-frequency-biased models overall show little, if any, increase in similarity to human neural representational geometry. Together, our results suggest that altered spatial-frequency reliance is likely an emergent property of learning more human-like representations rather than the primary mechanism by which neural alignment confers adversarial robustness, and motivate the need for future research examining representational properties beyond spatial-frequency profiles.