Goal Reasoning by Selecting Subgoals with Deep Q-Learning

TL;DR

This paper introduces a goal reasoning approach using Deep Q-Learning to select subgoals, significantly reducing planning time in online systems while maintaining plan quality across various game environments.

Contribution

It presents a CNN-based goal selection module trained with Deep Q-Learning, demonstrating improved efficiency and generalization in different planning domains and levels.

Findings

Reduces planning time compared to traditional methods.

Maintains high-quality plans across multiple game environments.

Shows strong generalization to new levels within the same domain.

Abstract

In this work we propose a goal reasoning method which learns to select subgoals with Deep Q-Learning in order to decrease the load of a planner when faced with scenarios with tight time restrictions, such as online execution systems. We have designed a CNN-based goal selection module and trained it on a standard video game environment, testing it on different games (planning domains) and levels (planning problems) to measure its generalization abilities. When comparing its performance with a satisfying planner, the results obtained show both approaches are able to find plans of good quality, but our method greatly decreases planning time. We conclude our approach can be successfully applied to different types of domains (games), and shows good generalization properties when evaluated on new levels (problems) of the same game (domain).