A method for the online construction of the set of states of a Markov Decision Process using Answer Set Programming

TL;DR

This paper introduces oASP(MDP), a novel method combining Markov Decision Processes and Answer Set Programming to dynamically construct state sets in non-stationary environments, enhancing policy adaptation.

Contribution

The paper proposes a new approach that integrates ASP with MDPs and RL to efficiently handle changing domains during decision-making.

Findings

oASP(MDP) effectively constructs state sets in non-stationary environments.

The method updates policies without disrupting RL action-value approximations.

Results demonstrate improved adaptability in dynamic domains.

Abstract

Non-stationary domains, that change in unpredicted ways, are a challenge for agents searching for optimal policies in sequential decision-making problems. This paper presents a combination of Markov Decision Processes (MDP) with Answer Set Programming (ASP), named {\em Online ASP for MDP} (oASP(MDP)), which is a method capable of constructing the set of domain states while the agent interacts with a changing environment. oASP(MDP) updates previously obtained policies, learnt by means of Reinforcement Learning (RL), using rules that represent the domain changes observed by the agent. These rules represent a set of domain constraints that are processed as ASP programs reducing the search space. Results show that oASP(MDP) is capable of finding solutions for problems in non-stationary domains without interfering with the action-value function approximation process.