Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents

TL;DR

This paper introduces DIAL, a method for adaptive compute in LLMs that learns the correct direction of gating signals, improving performance across diverse environments and models.

Contribution

It reveals the instability of fixed-direction gating signals and proposes a learned, environment-specific gating approach called DIAL.

Findings

DIAL outperforms fixed-direction baselines in success-cost trade-offs.

Gating signals can be unreliable due to conflicting interpretations of uncertainty.

Environment-specific gating directions improve adaptive compute effectiveness.

Abstract

Adaptive test-time compute for LLM agents aims to invoke extra computation only when it improves performance. Existing methods typically use confidence-, uncertainty-, or difficulty-based gates, assuming a fixed direction from the gating signal through compute need to the value of computation. This makes gating a utility-calibration problem: gating signals should align with whether extra computation improves the final outcome over the base policy. We show that this alignment is unstable: the same signal predicts rollout benefit in one setting and rollout harm in another, with reversals across environments and backbones even when the task is fixed. Wrong-direction gates can therefore worsen performance by precisely selecting harmful states. This reversal reflects a deeper distinction between compute need and compute suitability: a high uncertainty signal may indicate decision-difficult states where rollouts help compare alternatives, or intervention-unsuitable states where the current context does not support useful rollout-based improvement. Under this two-source model, fixed-direction gates are unreliable across heterogeneous settings. To address this, we propose DIAL (Direction-Informed Adaptive Learning), a sparse gate trained from signal-agnostic counterfactual exploration to learn the utility direction of state features per (environment, backbone). Across six environments and three backbones, DIAL yields a stronger overall success-cost trade-off than fixed-direction baselines.