Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations

TL;DR

This paper introduces a method to modify large language models' behavior by aligning behavioral and neural representations to identify steering vectors, enabling targeted influence without retraining.

Contribution

It presents a systematic approach to find steering vectors by aligning behavioral and neural representations, specifically for modulating risk preferences in LLMs.

Findings

Successfully modulated LLM risk-related outputs

Aligned neural and behavioral representations effectively

Steering vectors reliably influence model behavior

Abstract

Changing the behavior of large language models (LLMs) can be as straightforward as editing the Transformer's residual streams using appropriately constructed "steering vectors." These modifications to internal neural activations, a form of representation engineering, offer an effective and targeted means of influencing model behavior without retraining or fine-tuning the model. But how can such steering vectors be systematically identified? We propose a principled approach for uncovering steering vectors by aligning latent representations elicited through behavioral methods (specifically, Markov chain Monte Carlo with LLMs) with their neural counterparts. To evaluate this approach, we focus on extracting latent risk preferences from LLMs and steering their risk-related outputs using the aligned representations as steering vectors. We show that the resulting steering vectors successfully and reliably modulate LLM outputs in line with the targeted behavior.