Semantic Structure of Feature Space in Large Language Models

TL;DR

This paper reveals that the geometric relations between semantic features in large language models' hidden states closely resemble human psychological associations, highlighting the importance of feature space structure.

Contribution

It demonstrates that semantic features in large language models form meaningful low-dimensional subspaces that mirror human semantic associations and influence model ratings.

Findings

Semantic feature vectors correlate with human ratings

Semantic axes' similarities predict survey correlations

Semantic features lie on a low-dimensional subspace

Abstract

We show that the geometric relations between semantic features in large language models' hidden states closely mirror human psychological associations. We construct feature vectors corresponding to 360 words and project them on 32 semantic axes (e.g. beautiful-ugly, soft-hard), and find that these projections correlate highly with human ratings of those words on the respective semantic scales. Second, we find that the cosine similarities between the semantic axes themselves are highly predictive of the correlations between these scales in the survey. Third, we show that substantial variance across the 32 semantic axes lies on a low-dimensional subspace, reproducing patterns typical of human semantic associations. Finally, we demonstrate that steering a word on one semantic axis causes spillover effects on the model's rating of that word on other semantic scales proportionate to the cosine similarity between those semantic axes. These findings suggest that features should be understood not only in isolation but through their geometric relations and the meaningful subspaces they form.