The Effectiveness of Style Vectors for Steering Large Language Models: A Human Evaluation

TL;DR

This study evaluates activation steering for large language models, demonstrating its effectiveness in human-perceived emotional control, with reliable automatic scoring and improved consistency using LlaMA-3, supporting scalable model behavior steering.

Contribution

First human evaluation of activation steering for emotional tone in LLMs, showing reliable control and automatic quality proxy across different models and emotions.

Findings

Moderate steering amplifies target emotions effectively.

Automatic scoring correlates strongly with human ratings.

LlaMA-3 improves steering consistency and effects.

Abstract

Controlling the behavior of large language models (LLMs) at inference time is essential for aligning outputs with human abilities and safety requirements. \emph{Activation steering} provides a lightweight alternative to prompt engineering and fine-tuning by directly modifying internal activations to guide generation. This research advances the literature in three significant directions. First, while previous work demonstrated the technical feasibility of steering emotional tone using automated classifiers, this paper presents the first human evaluation of activation steering concerning the emotional tone of LLM outputs, collecting over 7,000 crowd-sourced ratings from 190 participants via Prolific ($n=190$). These ratings assess both perceived emotional intensity and overall text quality. Second, we find strong alignment between human and model-based quality ratings (mean $r=0.776$, range $0.157$--$0.985$), indicating automatic scoring can proxy perceived quality. Moderate steering strengths ($\lambda \approx 0.15$) reliably amplify target emotions while preserving comprehensibility, with the strongest effects for disgust ($\eta_p^2 = 0.616$) and fear ($\eta_p^2 = 0.540$), and minimal effects for surprise ($\eta_p^2 = 0.042$). Finally, upgrading from Alpaca to LlaMA-3 yielded more consistent steering with significant effects across emotions and strengths (all $p < 0.001$). Inter-rater reliability was high (ICC $= 0.71$--$0.87$), underscoring the robustness of the findings. These findings support activation-based control as a scalable method for steering LLM behavior across affective dimensions.