Improving the Accuracy and Robustness of CNNs Using a Deep CCA Neural Data Regularizer

TL;DR

This paper introduces a Deep CCA-based neural data regularizer for CNNs that enhances accuracy and robustness by making network representations more brain-like, outperforming previous methods.

Contribution

A novel neural data regularizer using Deep CCA that significantly improves CNN accuracy and robustness by aligning representations with primate visual cortex.

Findings

Significant increase in classification accuracy.

Enhanced robustness to adversarial attacks.

Better within-super-class accuracy.

Abstract

As convolutional neural networks (CNNs) become more accurate at object recognition, their representations become more similar to the primate visual system. This finding has inspired us and other researchers to ask if the implication also runs the other way: If CNN representations become more brain-like, does the network become more accurate? Previous attempts to address this question showed very modest gains in accuracy, owing in part to limitations of the regularization method. To overcome these limitations, we developed a new neural data regularizer for CNNs that uses Deep Canonical Correlation Analysis (DCCA) to optimize the resemblance of the CNN's image representations to that of the monkey visual cortex. Using this new neural data regularizer, we see much larger performance gains in both classification accuracy and within-super-class accuracy, as compared to the previous state-of-the-art neural data regularizers. These networks are also more robust to adversarial attacks than their unregularized counterparts. Together, these results confirm that neural data regularization can push CNN performance higher, and introduces a new method that obtains a larger performance boost.