Grounding Psychological Shape Space in Convolutional Neural Networks

TL;DR

This paper explores how convolutional neural networks can learn a generalizable, interpretable shape similarity space aligned with psychological data, improving over traditional methods that cannot handle novel stimuli.

Contribution

It introduces a neural network approach to map perceptual inputs to psychological shape spaces, enabling generalization to new stimuli and comparing different architectures and training regimes.

Findings

Multi-task learning with classification yields best results.

Performance depends on the dimensionality of the similarity space.

Neural networks outperform traditional multidimensional scaling methods.

Abstract

Shape information is crucial for human perception and cognition, and should therefore also play a role in cognitive AI systems. We employ the interdisciplinary framework of conceptual spaces, which proposes a geometric representation of conceptual knowledge through low-dimensional interpretable similarity spaces. These similarity spaces are often based on psychological dissimilarity ratings for a small set of stimuli, which are then transformed into a spatial representation by a technique called multidimensional scaling. Unfortunately, this approach is incapable of generalizing to novel stimuli. In this paper, we use convolutional neural networks to learn a generalizable mapping between perceptual inputs (pixels of grayscale line drawings) and a recently proposed psychological similarity space for the shape domain. We investigate different network architectures (classification network vs. autoencoder) and different training regimes (transfer learning vs. multi-task learning). Our results indicate that a classification-based multi-task learning scenario yields the best results, but that its performance is relatively sensitive to the dimensionality of the similarity space.