Words & Weights: Streamlining Multi-Turn Interactions via Co-Adaptation

TL;DR

ROSA2 introduces a joint optimization framework that co-adapts words and weights during test-time to improve multi-turn interactions with LLMs, reducing interaction turns and enhancing accuracy.

Contribution

It presents ROSA2, a novel method that combines textual gradients and parameter updates for synergistic test-time adaptation in multi-turn interactions.

Findings

ROSA2 outperforms baselines by 30% on MATH.

Reduces interaction turns by 40%.

Enhances alignment of LLMs with user intent.

Abstract

Test-time policy adaptation for multi-turn interactions (T2PAM) is essential for aligning Large Language Models (LLMs) with dynamic user needs during inference time. However, existing paradigms commonly treat test-time adaptation as a single-axis problem, either purely refining instructions (Prompt Engineering) or only adjusting weights (Test-Time Training), ignoring that interaction failures stem from a coupled mix of ambiguity and incapacity. We argue that these two optimization paths are not merely additive but synergistic: semantic clarity acts as a pre-conditioner for effective parameter updates. To this end, we propose ROSA2, a framework that reformulates interaction as a joint optimization problem over the heterogeneous space of Words and Weights. By mathematically decomposing the error signal, ROSA2 utilizes textual gradients to rectify intent ambiguity and parameter updates to bridge capability gaps. Theoretically, we prove that this co-adaptation strictly reduces the required parameter shift for convergence. Empirically, ROSA2 outperforms state-of-the-art baselines by 30% on MATH while reducing interaction turns by 40%, demonstrating that refining the context unlocks the true potential of parameter updates.