Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents
Quan M. Tran, Zhuo Huang, Wenbin Zhang, Bo Han, Koji Yatani, Masashi Sugiyama, Tongliang Liu

TL;DR
Bifrost is a training-free method that improves self-improving agents by adaptively steering past trajectories to new contexts, effectively bridging contextual gaps without fine-tuning.
Contribution
It introduces a novel context-trajectory correlation insight and a trajectory adaptation technique that aligns past experiences with new tasks at the representation level.
Findings
Bifrost outperforms existing methods across diverse benchmarks.
Trajectory adaptation at the representation level improves alignment.
The method effectively handles substantial context shifts.
Abstract
Autonomous agents excel in self-improvement through reflection and iterative refinement, which reuse successful task trajectories as in-context examples to assist subsequent reasoning. However, shifting across tasks often introduces a context mismatch. Hence, existing approaches either discard the trajectories or manipulate them using heuristics, leading to a non-negligible fine-tuning cost or unguaranteed performance. To bridge this gap, we reveal a context-trajectory correlation, where shifts of context are highly parallel with shifts of trajectory. Based on this finding, we propose BrIdge contextual gap FoR imprOvised trajectory STeering (Bifrost), a training-free method that leverages context differences to precisely guide the adaptation of previously solved trajectories towards the target task, mitigating the misalignment caused by context shifts. Our trajectory adaptation is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Multimodal Machine Learning Applications · AI-based Problem Solving and Planning
