Learning to Trust: Bayesian Adaptation to Varying Suggester Reliability in Sequential Decision Making

TL;DR

This paper presents a Bayesian framework enabling autonomous agents to adaptively trust external suggestions in uncertain, sequential decision tasks by learning suggester reliability and strategically requesting advice.

Contribution

It introduces a Bayesian approach that models and adapts to varying suggester reliability and incorporates an explicit 'ask' action for strategic suggestion requests.

Findings

Robust performance across different suggester qualities

Effective adaptation to changing reliability

Strategic suggestion requesting improves decision quality

Abstract

Autonomous agents operating in sequential decision-making tasks under uncertainty can benefit from external action suggestions, which provide valuable guidance but inherently vary in reliability. Existing methods for incorporating such advice typically assume static and known suggester quality parameters, limiting practical deployment. We introduce a framework that dynamically learns and adapts to varying suggester reliability in partially observable environments. First, we integrate suggester quality directly into the agent's belief representation, enabling agents to infer and adjust their reliance on suggestions through Bayesian inference over suggester types. Second, we introduce an explicit ``ask'' action allowing agents to strategically request suggestions at critical moments, balancing informational gains against acquisition costs. Experimental evaluation demonstrates robust performance across varying suggester qualities, adaptation to changing reliability, and strategic management of suggestion requests. This work provides a foundation for adaptive human-agent collaboration by addressing suggestion uncertainty in uncertain environments.