Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness

TL;DR

This paper introduces EVNets, a hybrid CNN architecture inspired by subcortical brain structures, which enhances robustness to visual perturbations and out-of-domain images beyond standard models.

Contribution

The paper presents a novel SubcorticalBlock architecture, inspired by neuroscience, that improves CNN robustness and biological alignment without specific optimization.

Findings

EVNets outperform baseline CNNs by 9.3% on robustness benchmarks.

SubcorticalBlock improves V1 alignment and models receptive field phenomena.

Combining EVNets with data augmentation yields a 6.2% further performance boost.

Abstract

Convolutional neural networks (CNNs) trained on object recognition achieve high task performance but continue to exhibit vulnerability under a range of visual perturbations and out-of-domain images, when compared with biological vision. Prior work has demonstrated that coupling a standard CNN with a front-end (VOneBlock) that mimics the primate primary visual cortex (V1) can improve overall model robustness. Expanding on this, we introduce Early Vision Networks (EVNets), a new class of hybrid CNNs that combine the VOneBlock with a novel SubcorticalBlock, whose architecture draws from computational models in neuroscience and is parameterized to maximize alignment with subcortical responses reported across multiple experimental studies. Without being optimized to do so, the assembly of the SubcorticalBlock with the VOneBlock improved V1 alignment across most standard V1 benchmarks, and better modeled extra-classical receptive field phenomena. In addition, EVNets exhibit stronger emergent shape bias and outperform the base CNN architecture by 9.3% on an aggregate benchmark of robustness evaluations, including adversarial perturbations, common corruptions, and domain shifts. Finally, we show that EVNets can be further improved when paired with a state-of-the-art data augmentation technique, surpassing the performance of the isolated data augmentation approach by 6.2% on our robustness benchmark. This result reveals complementary benefits between changes in architecture to better mimic biology and training-based machine learning approaches.