# Evaluating (and improving) the correspondence between deep neural networks and human representations

## TL;DR

This paper investigates how well deep neural networks mimic human mental representations, finding they align well with human similarity judgments but require adjustments to fully capture human perceptual structure, thereby enhancing psychological modeling.

## Contribution

It demonstrates that deep neural networks can predict human similarity judgments and introduces a convex optimization method to improve their alignment with human representations.

## Key findings

- Neural networks accurately predict human similarity judgments for natural images.
- A simple convex optimization correction improves neural network alignment with human representations.
- Enhanced representations enable better prediction of category learning difficulty.

## Abstract

Decades of psychological research have been aimed at modeling how people learn features and categories. The empirical validation of these theories is often based on artificial stimuli with simple representations. Recently, deep neural networks have reached or surpassed human accuracy on tasks such as identifying objects in natural images. These networks learn representations of real-world stimuli that can potentially be leveraged to capture psychological representations. We find that state-of-the-art object classification networks provide surprisingly accurate predictions of human similarity judgments for natural images, but fail to capture some of the structure represented by people. We show that a simple transformation that corrects these discrepancies can be obtained through convex optimization. We use the resulting representations to predict the difficulty of learning novel categories of natural images. Our results extend the scope of psychological experiments and computational modeling by enabling tractable use of large natural stimulus sets.