A hybrid learning agent for episodic learning tasks with unknown target distance

TL;DR

This paper introduces a hybrid reinforcement learning agent capable of efficiently learning in episodic tasks with unknown target distances by using a stochastic episode length strategy, demonstrating faster learning in simulations.

Contribution

The work extends the hybrid quantum-accessible reinforcement learning agent to handle scenarios with unknown episode lengths, a common real-world challenge.

Findings

Hybrid agent learns faster than classical agents in certain scenarios

Stochastic episode length strategy improves learning efficiency

Demonstrated effectiveness through simulation experiments

Abstract

The "hybrid agent for quantum-accessible reinforcement learning", as defined in (Hamann and W\"olk, 2022), provides a proven quasi-quadratic speedup and is experimentally tested. However, the standard version can only be applied to episodic learning tasks with fixed episode length. In many real-world applications, the information about the necessary number of steps within an episode to reach a defined target is not available in advance and especially before reaching the target for the first time. Furthermore, in such scenarios, classical agents have the advantage of observing at which step they reach the target. Whether the hybrid agent can provide an advantage in such learning scenarios was unknown so far. In this work, we introduce a hybrid agent with a stochastic episode length selection strategy to alleviate the need for knowledge about the necessary episode length. Through simulations, we test the adapted hybrid agent's performance versus classical counterparts. We find that the hybrid agent learns faster than corresponding classical learning agents in certain scenarios with unknown target distance and without fixed episode length.