What's in a prompt? Language models encode literary style in prompt embeddings

TL;DR

This paper demonstrates that large language models encode literary style information in prompt embeddings, revealing how stylistic features are geometrically represented and compressed in deep latent spaces.

Contribution

It shows that prompt embeddings encode stylistic features of literature, not just factual content, highlighting the models' sophisticated information processing capabilities.

Findings

Embeddings from different novels separate based on style, independent of content.

Ensembles from the same author are more entangled, indicating style encoding.

Deep representations contain intangible, stylistic information rather than just factual data.

Abstract

Large language models use high-dimensional latent spaces to encode and process textual information. Much work has investigated how the conceptual content of words translates into geometrical relationships between their vector representations. Fewer studies analyze how the cumulative information of an entire prompt becomes condensed into individual embeddings under the action of transformer layers. We use literary pieces to show that information about intangible, rather than factual, aspects of the prompt are contained in deep representations. We observe that short excerpts (10 - 100 tokens) from different novels separate in the latent space independently from what next-token prediction they converge towards. Ensembles from books from the same authors are much more entangled than across authors, suggesting that embeddings encode stylistic features. This geometry of style may have applications for authorship attribution and literary analysis, but most importantly reveals the sophistication of information processing and compression accomplished by language models.