ROAST: Rollout-based On-distribution Activation Steering Technique
Xuanbo Su, Hao Luo, Yingfang Zhang, Lijun Zhang

TL;DR
ROAST introduces a new activation steering method for large language models that estimates directions from on-distribution rollouts, improving robustness and performance without relying on off-distribution supervision or discrete masking.
Contribution
ROAST proposes a rollout-based on-distribution activation steering technique with continuous soft scaling and grouped normalization, enhancing robustness and effectiveness over prior methods.
Findings
ROAST improves task performance across models (e.g., +9.7% on GSM8K, +12.1% on TruthfulQA).
CSS better preserves activation energy and maintains semantic quality.
Grouped normalization balances contributions, leading to more reliable steering directions.
Abstract
Activation steering provides parameter-efficient control over large language models (LLMs) at inference time, but many methods rely on off-distribution supervision and discrete masking, leading to brittle interventions. We propose ROAST (Rollout-based On-distribution Activation Steering Technique), which estimates steering directions from the model's own on-distribution rollouts via ROC and avoids hard sparsification via Continuous Soft Scaling (CSS) and Grouped Mean Normalization. Our empirical analysis reveals that while activation magnitude correlates moderately with directional consistency, the variance in magnitude is significant and often disproportionate to semantic quality. This suggests that high-magnitude activations risk dominating the global steering direction if not properly normalized. To address this, ROAST employs grouped normalization to balance contributions across…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Domain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications
