Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order

TL;DR

This paper evaluates deep reinforcement learning algorithms on a task-oriented language grounding problem involving multiple sub-goals with non-linear execution order, highlighting challenges and the impact of language structure on success rates.

Contribution

It introduces a simple instructional language with non-linear order connectors for GridWorld and analyzes the performance of deep RL algorithms on this setup.

Findings

Non-linear order connectors increase success rates 2-3 times for complex instructions.

Success rates remain below 20% despite improvements.

Gated-Attention does not outperform simple concatenation in this context.

Abstract

In this work, we analyze the performance of general deep reinforcement learning algorithms for a task-oriented language grounding problem, where language input contains multiple sub-goals and their order of execution is non-linear. We generate a simple instructional language for the GridWorld environment, that is built around three language elements (order connectors) defining the order of execution: one linear - "comma" and two non-linear - "but first", "but before". We apply one of the deep reinforcement learning baselines - Double DQN with frame stacking and ablate several extensions such as Prioritized Experience Replay and Gated-Attention architecture. Our results show that the introduction of non-linear order connectors improves the success rate on instructions with a higher number of sub-goals in 2-3 times, but it still does not exceed 20%. Also, we observe that the usage of Gated-Attention provides no competitive advantage against concatenation in this setting. Source code and experiments' results are available at https://github.com/vkurenkov/language-grounding-multigoal