Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models

TL;DR

This paper introduces training-free inference-time strategies to improve large audio-language models' reasoning capabilities, achieving accuracy gains and demonstrating cross-modal transfer with high data efficiency.

Contribution

It proposes novel model steering methods for LALMs that enhance reasoning without additional training, including cross-modal transfer techniques.

Findings

Accuracy improved by up to 4.4% over standard CoT prompting

Cross-modal transfer enables effective reasoning with minimal text samples

Hyperparameter sensitivity analysis shows robustness of the methods

Abstract

Chain-of-thought (CoT) prompting has been extended to large audio-language models (LALMs) to elicit reasoning, yet enhancing its effectiveness without training remains challenging. We study inference-time model steering as a training-free approach to improve LALM reasoning. We introduce three strategies using diverse information sources and evaluate them across four LALMs and four benchmarks. Results show general accuracy gains up to 4.4% over CoT prompting. Notably, we identify a cross-modal transfer where steering vectors derived from few text samples effectively guide speech-based reasoning, demonstrating high data efficiency. We also examine hyperparameter sensitivity to understand the robustness of these approaches. Our findings position model steering as a practical direction for strengthening LALM reasoning.