Context-Generative Default Policy for Bounded Rational Agent

TL;DR

This paper introduces a context-generative default policy for bounded rational agents that predicts unobserved environment regions to improve decision-making and adaptability in unknown environments, using diffusion models and sampling-based planning.

Contribution

It proposes a novel adaptive default policy framework that leverages environment context and imagined maps to enhance robot navigation in unseen environments.

Findings

Outperforms baseline methods in obstacle avoidance.

Demonstrates effective adaptation in real-world drone experiments.

Utilizes diffusion models for environment prediction.

Abstract

Bounded rational agents often make decisions by evaluating a finite selection of choices, typically derived from a reference point termed the $`$default policy,' based on previous experience. However, the inherent rigidity of the static default policy presents significant challenges for agents when operating in unknown environment, that are not included in agent's prior knowledge. In this work, we introduce a context-generative default policy that leverages the region observed by the robot to predict unobserved part of the environment, thereby enabling the robot to adaptively adjust its default policy based on both the actual observed map and the $\textit{imagined}$ unobserved map. Furthermore, the adaptive nature of the bounded rationality framework enables the robot to manage unreliable or incorrect imaginations by selectively sampling a few trajectories in the vicinity of the default policy. Our approach utilizes a diffusion model for map prediction and a sampling-based planning with B-spline trajectory optimization to generate the default policy. Extensive evaluations reveal that the context-generative policy outperforms the baseline methods in identifying and avoiding unseen obstacles. Additionally, real-world experiments conducted with the Crazyflie drones demonstrate the adaptability of our proposed method, even when acting in environments outside the domain of the training distribution.