The Inner Sentiments of a Thought
Chris Gagne, Peter Dayan

TL;DR
This paper investigates how large language models implicitly encode and can be used to analyze and generate sentences with specific sentiments, revealing insights into their internal emotional representations.
Contribution
It introduces methods to predict and analyze sentiment distributions within LLMs, providing new tools for understanding and manipulating the model's emotional content.
Findings
Predictors of sentiment distributions are well calibrated.
Sentiment predictors can analyze how conjunctions affect emotional tone.
Distributional predictions enable generation of sentences with extreme sentiments.
Abstract
Transformer-based large-scale language models (LLMs) are able to generate highly realistic text. They are duly able to express, and at least implicitly represent, a wide range of sentiments and color, from the obvious, such as valence and arousal to the subtle, such as determination and admiration. We provide a first exploration of these representations and how they can be used for understanding the inner sentimental workings of single sentences. We train predictors of the quantiles of the distributions of final sentiments of sentences from the hidden representations of an LLM applied to prefixes of increasing lengths. After showing that predictors of distributions of valence, determination, admiration, anxiety and annoyance are well calibrated, we provide examples of using these predictors for analyzing sentences, illustrating, for instance, how even ordinary conjunctions (e.g., "but")…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMental Health via Writing · Topic Modeling · Machine Learning in Healthcare
