Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks and Autoregressive Policy Decomposition

TL;DR

This paper introduces a domain-independent deep reinforcement learning framework using graph neural networks and autoregressive policy decomposition, enabling effective learning and zero-shot generalization in relational problems with variable state and action spaces.

Contribution

The authors propose a novel RL framework that leverages graph neural networks and autoregressive policies to handle relational problems with variable sizes, demonstrating broad applicability and zero-shot generalization.

Findings

Effective zero-shot generalization across different problem sizes.

Broad applicability demonstrated in three distinct domains.

Outperforms existing methods in relational RL tasks.

Abstract

We focus on reinforcement learning (RL) in relational problems that are naturally defined in terms of objects, their relations, and object-centric actions. These problems are characterized by variable state and action spaces, and finding a fixed-length representation, required by most existing RL methods, is difficult, if not impossible. We present a deep RL framework based on graph neural networks and auto-regressive policy decomposition that naturally works with these problems and is completely domain-independent. We demonstrate the framework's broad applicability in three distinct domains and show impressive zero-shot generalization over different problem sizes.