Hint assisted reinforcement learning: an application in radio astronomy

TL;DR

This paper introduces a hint-assisted reinforcement learning approach for radio astronomy, leveraging environment-generated hints to improve sample efficiency and reduce the complexity of model construction.

Contribution

It proposes a novel method that integrates hints into reinforcement learning, specifically modifying the soft actor critic algorithm for better efficiency in radio astronomy tasks.

Findings

Increased sample efficiency with hint integration

Effective use of hints reduces model complexity

Method outperforms traditional model-free approaches

Abstract

Model based reinforcement learning has proven to be more sample efficient than model free methods. On the other hand, the construction of a dynamics model in model based reinforcement learning has increased complexity. Data processing tasks in radio astronomy are such situations where the original problem which is being solved by reinforcement learning itself is the creation of a model. Fortunately, many methods based on heuristics or signal processing do exist to perform the same tasks and we can leverage them to propose the best action to take, or in other words, to provide a `hint'. We propose to use `hints' generated by the environment as an aid to the reinforcement learning process mitigating the complexity of model construction. We modify the soft actor critic algorithm to use hints and use the alternating direction method of multipliers algorithm with inequality constraints to train the agent. Results in several environments show that we get the increased sample efficiency by using hints as compared to model free methods.