Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness

TL;DR

This paper introduces a pixel-based similarity regularizer for CNNs inspired by neural data, which improves adversarial robustness without requiring neural recordings, making it more practical and easily integrable.

Contribution

A novel data-driven regularizer replacing neural similarity with pixel-based similarity, maintaining robustness benefits while simplifying implementation.

Findings

Pixel-based regularizer achieves robustness comparable to neural data-based methods.

The approach is lightweight and easily integrated into standard CNN training pipelines.

Does not surpass state-of-the-art defenses but demonstrates the potential of brain-inspired, data-driven methods.

Abstract

Convolutional Neural Networks (CNNs) excel in many visual tasks but remain susceptible to adversarial attacks-imperceptible perturbations that degrade performance. Prior research reveals that brain-inspired regularizers, derived from neural recordings, can bolster CNN robustness; however, reliance on specialized data limits practical adoption. We revisit a regularizer proposed by Li et al. (2019) that aligns CNN representations with neural representational similarity structures and introduce a data-driven variant. Instead of a neural recording-based similarity, our method computes a pixel-based similarity directly from images. This substitution retains the original biologically motivated loss formulation, preserving its robustness benefits while removing the need for neural measurements or task-specific augmentations. Notably, this data-driven variant provides the same robustness improvements observed with neural data. Our approach is lightweight and integrates easily into standard pipelines. Although we do not surpass cutting-edge specialized defenses, we show that neural representational insights can be leveraged without direct recordings. This underscores the promise of robust yet simple methods rooted in brain-inspired principles, even without specialized data, and raises the possibility that further integrating these insights could push performance closer to human levels without resorting to complex, specialized pipelines.