In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specifications

TL;DR

This paper introduces a neural architecture with inductive biases that enables agents to learn latent goal representations, improving generalization in multi-task, out-of-distribution instructions expressed in temporal logic using deep reinforcement learning.

Contribution

It proposes a novel deep learning configuration with inductive biases for latent goal generation within a neuro-symbolic framework, enhancing OOD generalization in TL tasks.

Findings

Latent-goal networks outperform SOTA architectures in OOD generalization.

Inductive biases improve the neural network's ability to represent goals.

Neuro-symbolic framework effectively executes multi-task TL instructions.

Abstract

We address the problem of building agents whose goal is to learn to execute out-of distribution (OOD) multi-task instructions expressed in temporal logic (TL) by using deep reinforcement learning (DRL). Recent works provided evidence that the agent's neural architecture is a key feature when DRL agents are learning to solve OOD tasks in TL. Yet, the studies on this topic are still in their infancy. In this work, we propose a new deep learning configuration with inductive biases that lead agents to generate latent representations of their current goal, yielding a stronger generalization performance. We use these latent-goal networks within a neuro-symbolic framework that executes multi-task formally-defined instructions and contrast the performance of the proposed neural networks against employing different state-of-the-art (SOTA) architectures when generalizing to unseen instructions in OOD environments.