A Neurobiological Evaluation Metric for Neural Network Model Search

TL;DR

This paper introduces a human-model similarity (HMS) metric that measures how closely neural network activation patterns resemble human brain responses, and finds that higher HMS correlates with better visual task performance.

Contribution

The paper proposes a novel HMS metric based on fMRI and network activation similarity, linking brain-like behavior to improved neural network performance.

Findings

Higher HMS correlates with better task performance.

HMS can serve as an early-stopping criterion during training.

Networks with high HMS exhibit superior generalization.

Abstract

Neuroscience theory posits that the brain's visual system coarsely identifies broad object categories via neural activation patterns, with similar objects producing similar neural responses. Artificial neural networks also have internal activation behavior in response to stimuli. We hypothesize that networks exhibiting brain-like activation behavior will demonstrate brain-like characteristics, e.g., stronger generalization capabilities. In this paper we introduce a human-model similarity (HMS) metric, which quantifies the similarity of human fMRI and network activation behavior. To calculate HMS, representational dissimilarity matrices (RDMs) are created as abstractions of activation behavior, measured by the correlations of activations to stimulus pairs. HMS is then the correlation between the fMRI RDM and the neural network RDM across all stimulus pairs. We test the metric on unsupervised predictive coding networks, which specifically model visual perception, and assess the metric for statistical significance over a large range of hyperparameters. Our experiments show that networks with increased human-model similarity are correlated with better performance on two computer vision tasks: next frame prediction and object matching accuracy. Further, HMS identifies networks with high performance on both tasks. An unexpected secondary finding is that the metric can be employed during training as an early-stopping mechanism.