A Neurobiologically Motivated Analysis of Distributional Semantic Models

TL;DR

This paper investigates what types of semantic information are encoded in distributional word vectors by comparing them to neurobiologically motivated brain-based vectors, revealing strengths in social and cognitive information encoding.

Contribution

It introduces a method to analyze and interpret the internal information of word vectors using brain-based conceptual representations, bridging computational models and neurobiological insights.

Findings

Social and cognitive information are well encoded in word vectors.

Emotional information is poorly represented in text-based vectors.

Analysis supports embodied theories for abstract concepts.

Abstract

The pervasive use of distributional semantic models or word embeddings in a variety of research fields is due to their remarkable ability to represent the meanings of words for both practical application and cognitive modeling. However, little has been known about what kind of information is encoded in text-based word vectors. This lack of understanding is particularly problematic when word vectors are regarded as a model of semantic representation for abstract concepts. This paper attempts to reveal the internal information of distributional word vectors by the analysis using Binder et al.'s (2016) brain-based vectors, explicitly structured conceptual representations based on neurobiologically motivated attributes. In the analysis, the mapping from text-based vectors to brain-based vectors is trained and prediction performance is evaluated by comparing the estimated and original brain-based vectors. The analysis demonstrates that social and cognitive information is better encoded in text-based word vectors, but emotional information is not. This result is discussed in terms of embodied theories for abstract concepts.