Self-directed Learning of Action Models using Exploratory Planning

TL;DR

This paper introduces a novel exploratory planning agent capable of learning action models in unknown domains without prior expert guidance, using new representations and planning strategies to improve exploration and learning efficiency.

Contribution

It presents a new representation called Lifted Linked Clauses, an exploration action selection method, and an exploration planner that enhances learning in unknown environments.

Findings

Lifted Linked Clauses improve exploration efficiency.

The agent effectively learns action models without expert traces.

Empirical results show superior performance over baseline agents.

Abstract

Complex, real-world domains may not be fully modeled for an agent, especially if the agent has never operated in the domain before. The agent's ability to effectively plan and act in such a domain is influenced by its knowledge of when it can perform specific actions and the effects of those actions. We describe a novel exploratory planning agent that is capable of learning action preconditions and effects without expert traces or a given goal. The agent's architecture allows it to perform both exploratory actions as well as goal-directed actions, which opens up important considerations for how exploratory planning and goal planning should be controlled, as well as how the agent's behavior should be explained to any teammates it may have. The contributions of this work include a new representation for contexts called Lifted Linked Clauses, a novel exploration action selection approach using these clauses, an exploration planner that uses lifted linked clauses as goals in order to reach new states, and an empirical evaluation in a scenario from an exploration-focused video game demonstrating that lifted linked clauses improve exploration and action model learning against non-planning baseline agents.