Q-Search Trees: An Information-Theoretic Approach Towards Hierarchical Abstractions for Agents with Computational Limitations

TL;DR

This paper introduces a novel information-theoretic framework for creating hierarchical graph abstractions tailored to an agent's limited computational resources, enhancing decision-making efficiency.

Contribution

It formulates a new optimization problem for tree-based abstractions and proposes two algorithms to solve it, linking computational constraints with abstraction quality.

Findings

The framework effectively generates meaningful hierarchies in complex environments.

Proposed algorithms produce solutions with provable quality bounds.

The approach connects signal compression principles with decision-making abstractions.

Abstract

In this paper, we develop a framework to obtain graph abstractions for decision-making by an agent where the abstractions emerge as a function of the agent's limited computational resources. We discuss the connection of the proposed approach with information-theoretic signal compression, and formulate a novel optimization problem to obtain tree-based abstractions as a function of the agent's computational resources. The structural properties of the new problem are discussed in detail, and two algorithmic approaches are proposed to obtain solutions to this optimization problem. We discuss the quality of, and prove relationships between, solutions obtained by the two proposed algorithms. The framework is demonstrated to generate a hierarchy of abstractions for a non-trivial environment.