A Comparative Study of Dynamic Programming and Reinforcement Learning in Finite Horizon Dynamic Pricing

TL;DR

This study systematically compares Fitted Dynamic Programming and Reinforcement Learning in finite-horizon dynamic pricing, analyzing their performance across complex, multi-dimensional environments with various demand structures and constraints.

Contribution

It extends prior work by applying DP in richer, multi-product settings and provides a comprehensive performance analysis against RL methods.

Findings

DP performs well in low-dimensional settings but faces scalability issues.

RL demonstrates better adaptability in complex, multi-typology environments.

Trade-offs exist between optimization accuracy and computational efficiency.

Abstract

This paper provides a systematic comparison between Fitted Dynamic Programming (DP), where demand is estimated from data, and Reinforcement Learning (RL) methods in finite-horizon dynamic pricing problems. We analyze their performance across environments of increasing structural complexity, ranging from a single typology benchmark to multi-typology settings with heterogeneous demand and inter-temporal revenue constraints. Unlike simplified comparisons that restrict DP to low-dimensional settings, we apply dynamic programming in richer, multi-dimensional environments with multiple product types and constraints. We evaluate revenue performance, stability, constraint satisfaction behavior, and computational scaling, highlighting the trade-offs between explicit expectation-based optimization and trajectory-based learning.