BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents

TL;DR

This paper introduces BrowseConf, a confidence-guided test-time scaling method for web agents that improves answer reliability and reduces token usage by leveraging confidence scores in multi-turn interactions.

Contribution

It presents a novel approach to utilize confidence scores for dynamic answer quality control in multi-turn LLM-based search agents, addressing a gap in existing research.

Findings

Models show higher accuracy at high confidence levels.

Proposed TTS methods reduce token consumption significantly.

TTS achieves competitive performance with fixed budget baselines.

Abstract

Confidence in LLMs is a useful indicator of model uncertainty and answer reliability. Existing work mainly focused on single-turn scenarios, while research on confidence in complex multi-turn interactions is limited. In this paper, we investigate whether LLM-based search agents have the ability to communicate their own confidence through verbalized confidence scores after long sequences of actions, a significantly more challenging task compared to outputting confidence in a single interaction. Experimenting on open-source agentic models, we first find that models exhibit much higher task accuracy at high confidence while having near-zero accuracy when confidence is low. Based on this observation, we propose Test-Time Scaling (TTS) methods that use confidence scores to determine answer quality, encourage the model to try again until reaching a satisfactory confidence level. Results show that our proposed methods significantly reduce token consumption while demonstrating competitive performance compared to baseline fixed budget TTS methods.