On Avoidance Learning with Partial Observability

TL;DR

This paper introduces A-learning, a parameter-free algorithm for agents with partial observability and non-determinism to reliably avoid aversive signals, demonstrating fixpoint convergence.

Contribution

It proposes a novel, simple, and parameter-free learning algorithm for avoidance tasks under partial observability and non-determinism, with proven convergence.

Findings

A-learning converges to a fixpoint of allowed feature-action pairs.

The algorithm is easy to implement and does not require parameter tuning.

It effectively handles partial information and non-deterministic action outcomes.

Abstract

We study a framework where agents have to avoid aversive signals. The agents are given only partial information, in the form of features that are projections of task states. Additionally, the agents have to cope with non-determinism, defined as unpredictability on the way that actions are executed. The goal of each agent is to define its behavior based on feature-action pairs that reliably avoid aversive signals. We study a learning algorithm, called A-learning, that exhibits fixpoint convergence, where the belief of the allowed feature-action pairs eventually becomes fixed. A-learning is parameter-free and easy to implement.