Achieving Stable Training of Reinforcement Learning Agents in Bimodal Environments through Batch Learning

TL;DR

This paper introduces a batch learning approach to improve the stability and effectiveness of reinforcement learning agents in bimodal, stochastic environments, especially for pricing applications.

Contribution

A novel batch update method for tabular Q-learning tailored to bimodal stochastic environments, demonstrating improved stability and performance over traditional methods.

Findings

Batch learning agents outperform traditional agents in effectiveness.

Batch agents show increased resilience to environment fluctuations.

Potential for practical deployment in industrial pricing applications.

Abstract

Bimodal, stochastic environments present a challenge to typical Reinforcement Learning problems. This problem is one that is surprisingly common in real world applications, being particularly applicable to pricing problems. In this paper we present a novel learning approach to the tabular Q-learning algorithm, tailored to tackling these specific challenges by using batch updates. A simulation of pricing problem is used as a testbed to compare a typically updated agent with a batch learning agent. The batch learning agents are shown to be both more effective than the typically-trained agents, and to be more resilient to the fluctuations in a large stochastic environment. This work has a significant potential to enable practical, industrial deployment of Reinforcement Learning in the context of pricing and others.