Solving the Dual Problems of Dynamic Programs via Regression

TL;DR

This paper introduces a regression-based framework for approximating optimal dual penalties in dynamic programming, enabling efficient and feasible dual bounds without nested simulation, demonstrated on a high-dimensional trading problem.

Contribution

It develops a non-nested regression approach to approximate dual penalties, improving computational efficiency and tractability in solving complex dynamic programs.

Findings

Framework provides valid dual bounds efficiently.

Approximations are feasible dual penalties.

Effective in high-dimensional trading applications.

Abstract

In recent years, information relaxation and duality in dynamic programs have been studied extensively, and the resulted primal-dual approach has become a powerful procedure in solving dynamic programs by providing lower-upper bounds on the optimal value function. Theoretically, with the so called value-based optimal dual penalty, the optimal value function could be recovered exactly via strong duality. However, in practice, obtaining tight dual bounds usually requires good approximations of the optimal dual penalty, which could be time-consuming due to the conditional expectations that need to be estimated via nested simulation. In this paper, we will develop a framework of regression approach to approximating the optimal dual penalty in a non-nested manner, by exploring the structure of the function space consisting of all feasible dual penalties. The resulted approximations maintain to be feasible dual penalties, and thus yield valid dual bounds on the optimal value function. We show that the proposed framework is computationally efficient, and the resulted dual penalties lead to numerically tractable dual problems. Finally, we apply the framework to a high-dimensional dynamic trading problem to demonstrate its effectiveness in solving the dual problems of complex dynamic programs.