$\epsilon$-Policy Gradient for Online Pricing

TL;DR

This paper introduces an $$-policy gradient algorithm that combines model-based and model-free reinforcement learning for online pricing, achieving near-optimal regret bounds by balancing exploration and exploitation.

Contribution

It proposes a novel $$-policy gradient method that extends $$-greedy algorithms with gradient-based learning and analyzes its regret performance in online pricing.

Findings

Achieves expected regret of order $( ext{T})$ with logarithmic factors.

Balances exploration and exploitation effectively in online pricing.

Provides theoretical analysis of regret bounds for the proposed algorithm.

Abstract

Combining model-based and model-free reinforcement learning approaches, this paper proposes and analyzes an $\epsilon$-policy gradient algorithm for the online pricing learning task. The algorithm extends $\epsilon$-greedy algorithm by replacing greedy exploitation with gradient descent step and facilitates learning via model inference. We optimize the regret of the proposed algorithm by quantifying the exploration cost in terms of the exploration probability $\epsilon$ and the exploitation cost in terms of the gradient descent optimization and gradient estimation errors. The algorithm achieves an expected regret of order $\mathcal{O}(\sqrt{T})$ (up to a logarithmic factor) over $T$ trials.