Decision-Making Among Bounded Rational Agents

TL;DR

This paper introduces a game-theoretic framework incorporating bounded rationality via information theory, enabling robots to predict and adapt to sub-optimal behaviors of other agents under computational constraints.

Contribution

It proposes a novel bounded rationality model using KL-divergence to represent agents' limited information processing, with an efficient importance sampling solution.

Findings

Framework effectively models various rationality levels of agents.

Robots can compute strategies considering other agents' sub-optimal behaviors.

Experimental results show improved navigation in multi-robot scenarios.

Abstract

When robots share the same workspace with other intelligent agents (e.g., other robots or humans), they must be able to reason about the behaviors of their neighboring agents while accomplishing the designated tasks. In practice, frequently, agents do not exhibit absolutely rational behavior due to their limited computational resources. Thus, predicting the optimal agent behaviors is undesirable (because it demands prohibitive computational resources) and undesirable (because the prediction may be wrong). Motivated by this observation, we remove the assumption of perfectly rational agents and propose incorporating the concept of bounded rationality from an information-theoretic view into the game-theoretic framework. This allows the robots to reason other agents' sub-optimal behaviors and act accordingly under their computational constraints. Specifically, bounded rationality directly models the agent's information processing ability, which is represented as the KL-divergence between nominal and optimized stochastic policies, and the solution to the bounded-optimal policy can be obtained by an efficient importance sampling approach. Using both simulated and real-world experiments in multi-robot navigation tasks, we demonstrate that the resulting framework allows the robots to reason about different levels of rational behaviors of other agents and compute a reasonable strategy under its computational constraint.