Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations

TL;DR

This paper introduces SENSE, a model that predicts sensorimotor norms from lexical embeddings, linking language models with human sensory-motor experiences through behavioral validation.

Contribution

It presents a novel projection model connecting word embeddings with sensorimotor norms and validates it with human behavioral data.

Findings

Significant correlations between SENSE predictions and human sensorimotor association ratings.

Behavioral study with 281 participants supports the model's validity.

Phonosthemic patterns found in nonce words suggest a link between phonology and sensorimotor norms.

Abstract

While word embeddings derive meaning from co-occurrence patterns, human language understanding is grounded in sensory and motor experience. We present $\text{SENSE}$ $(\textbf{S}\text{ensorimotor }$ $\textbf{E}\text{mbedding }$ $\textbf{N}\text{orm }$ $\textbf{S}\text{coring }$ $\textbf{E}\text{ngine})$, a learned projection model that predicts Lancaster sensorimotor norms from word lexical embeddings. We also conducted a behavioral study where 281 participants selected which among candidate nonce words evoked specific sensorimotor associations, finding statistically significant correlations between human selection rates and $\text{SENSE}$ ratings across 6 of the 11 modalities. Sublexical analysis of these nonce words selection rates revealed systematic phonosthemic patterns for the interoceptive norm, suggesting a path towards computationally proposing candidate phonosthemes from text data.