Learning to Operate in Open Worlds by Adapting Planning Models

TL;DR

This paper presents a method for planning agents to detect and adapt to novelties in open worlds by revising their domain models based on observed divergences, demonstrated on the CartPole benchmark.

Contribution

It introduces a novel approach for agents to identify and adapt to environmental novelties by revising their models through heuristics-guided search, enhancing robustness in open worlds.

Findings

Effective detection of novelties through divergence measurement

Rapid adaptation to a class of novelties in CartPole

Model revisions are interpretable and efficient

Abstract

Planning agents are ill-equipped to act in novel situations in which their domain model no longer accurately represents the world. We introduce an approach for such agents operating in open worlds that detects the presence of novelties and effectively adapts their domain models and consequent action selection. It uses observations of action execution and measures their divergence from what is expected, according to the environment model, to infer existence of a novelty. Then, it revises the model through a heuristics-guided search over model changes. We report empirical evaluations on the CartPole problem, a standard Reinforcement Learning (RL) benchmark. The results show that our approach can deal with a class of novelties very quickly and in an interpretable fashion.