From model-based learning to model-free behaviour with Meta-Interpretive Learning

TL;DR

This paper presents a method combining model-based and model-free learning using Meta-Interpretive Learning to create autonomous agents capable of solving planning problems in novel environments, demonstrating their equivalence in problem-solving.

Contribution

It introduces a novel approach that uses Meta-Interpretive Learning to train a model-free controller from a model-based solver, unifying planning and acting capabilities.

Findings

Controller solves all problems the Solver can solve

Demonstrates equivalence in problem-solving ability

Applicable to maze and lake map environments

Abstract

A "model" is a theory that describes the state of an environment and the effects of an agent's decisions on the environment. A model-based agent can use its model to predict the effects of its future actions and so plan ahead, but must know the state of the environment. A model-free agent cannot plan, but can act without a model and without completely observing the environment. An autonomous agent capable of acting independently in novel environments must combine both sets of capabilities. We show how to create such an agent with Meta-Interpretive Learning used to learn a model-based Solver used to train a model-free Controller that can solve the same planning problems as the Solver. We demonstrate the equivalence in problem-solving ability of the two agents on grid navigation problems in two kinds of environment: randomly generated mazes, and lake maps with wide open areas. We find that all navigation problems solved by the Solver are also solved by the Controller, indicating the two are equivalent.