Bounded Rational Decision-Making with Adaptive Neural Network Priors

TL;DR

This paper explores how neural networks can serve as adaptive priors in bounded rational decision-making, optimizing resource use in sample-based processes like MCMC, and demonstrates this approach on toy examples.

Contribution

It introduces a method for jointly optimizing neural network priors with decision-making processes, enhancing efficiency in bounded rational models.

Findings

Neural network priors can be effectively optimized alongside decision processes.

Adaptive priors improve sample efficiency in toy decision-making scenarios.

The approach demonstrates potential for more efficient bounded rational models.

Abstract

Bounded rationality investigates utility-optimizing decision-makers with limited information-processing power. In particular, information theoretic bounded rationality models formalize resource constraints abstractly in terms of relative Shannon information, namely the Kullback-Leibler Divergence between the agents' prior and posterior policy. Between prior and posterior lies an anytime deliberation process that can be instantiated by sample-based evaluations of the utility function through Markov Chain Monte Carlo (MCMC) optimization. The most simple model assumes a fixed prior and can relate abstract information-theoretic processing costs to the number of sample evaluations. However, more advanced models would also address the question of learning, that is how the prior is adapted over time such that generated prior proposals become more efficient. In this work we investigate generative neural networks as priors that are optimized concurrently with anytime sample-based decision-making processes such as MCMC. We evaluate this approach on toy examples.