RL as Regressor: A Reinforcement Learning Approach for Function Approximation

TL;DR

This paper presents a novel approach to regression by framing it as a reinforcement learning problem, allowing for flexible objective functions and leveraging RL algorithms for function approximation.

Contribution

It introduces a reinforcement learning framework for regression, demonstrating its effectiveness through a case study with an Actor-Critic agent on a noisy sine wave.

Findings

RL-based regression successfully models complex functions

Enhanced flexibility in defining custom objectives

Improved learning with experience replay and network enhancements

Abstract

Standard regression techniques, while powerful, are often constrained by predefined, differentiable loss functions such as mean squared error. These functions may not fully capture the desired behavior of a system, especially when dealing with asymmetric costs or complex, non-differentiable objectives. In this paper, we explore an alternative paradigm: framing regression as a Reinforcement Learning (RL) problem. We demonstrate this by treating a model's prediction as an action and defining a custom reward signal based on the prediction error, and we can leverage powerful RL algorithms to perform function approximation. Through a progressive case study of learning a noisy sine wave, we illustrate the development of an Actor-Critic agent, iteratively enhancing it with Prioritized Experience Replay, increased network capacity, and positional encoding to enable a capable RL agent for this regression task. Our results show that the RL framework not only successfully solves the regression problem but also offers enhanced flexibility in defining objectives and guiding the learning process.