Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents

TL;DR

Bifrost is a training-free method that improves self-improving agents by adaptively steering past trajectories to new contexts, effectively bridging contextual gaps without fine-tuning.

Contribution

It introduces a novel context-trajectory correlation insight and a trajectory adaptation technique that aligns past experiences with new tasks at the representation level.

Findings

Bifrost outperforms existing methods across diverse benchmarks.

Trajectory adaptation at the representation level improves alignment.

The method effectively handles substantial context shifts.

Abstract

Autonomous agents excel in self-improvement through reflection and iterative refinement, which reuse successful task trajectories as in-context examples to assist subsequent reasoning. However, shifting across tasks often introduces a context mismatch. Hence, existing approaches either discard the trajectories or manipulate them using heuristics, leading to a non-negligible fine-tuning cost or unguaranteed performance. To bridge this gap, we reveal a context-trajectory correlation, where shifts of context are highly parallel with shifts of trajectory. Based on this finding, we propose BrIdge contextual gap FoR imprOvised trajectory STeering (Bifrost), a training-free method that leverages context differences to precisely guide the adaptation of previously solved trajectories towards the target task, mitigating the misalignment caused by context shifts. Our trajectory adaptation is conducted at the representation level using agent hidden states, ensuring trajectory transformation accurately aligns with the target context in a shared space. Across diverse benchmarks, Bifrost consistently outperforms existing trajectory reuse and finetuned self-improvement methods, demonstrating that agents can effectively leverage past experiences despite substantial context shifts.