Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

TL;DR

This paper introduces a method combining graph neural networks with reinforcement learning to generate vehicle behaviors in semantic environments, overcoming limitations of vector-based inputs by handling variable object counts and orders.

Contribution

The paper proposes integrating graph neural networks with actor-critic reinforcement learning for behavior generation in semantic environments, enabling invariant and relational object processing.

Findings

Graph neural networks handle varying numbers of vehicles effectively.

The approach outperforms conventional methods in lane-change scenarios.

GNNs explicitly utilize relational information for better decision-making.

Abstract

Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network to have a pre-defined input-size -- in semantic environments this means assuming the maximum number of vehicles. Additionally, this vectorial representation is not invariant to the order and number of vehicles. To mitigate the above-stated disadvantages, we propose combining graph neural networks with actor-critic reinforcement learning. As graph neural networks apply the same network to every vehicle and aggregate incoming edge information, they are invariant to the number and order of vehicles. This makes them ideal candidates to be used as networks in semantic environments -- environments consisting of objects lists. Graph neural networks exhibit some other advantages that make them favorable to be used in semantic environments. The relational information is explicitly given and does not have to be inferred. Moreover, graph neural networks propagate information through the network and can gather higher-degree information. We demonstrate our approach using a highway lane-change scenario and compare the performance of graph neural networks to conventional ones. We show that graph neural networks are capable of handling scenarios with a varying number and order of vehicles during training and application.