An Alternating Approach to Approximate Dynamic Programming

TL;DR

This paper introduces an alternating approximate dynamic programming method that reduces constraints and variables, employs kernel approximation for basis functions, and demonstrates its effectiveness through option pricing applications.

Contribution

The paper proposes a novel alternating ADP approach with fewer constraints and variables, using kernel approximation for nonlinear function learning in large-scale MDPs.

Findings

American call option not exercised early without dividends

Method efficiently handles high-dimensional option pricing

Provides bounds on option prices using benchmark comparisons

Abstract

In this paper, we give a new approximate dynamic programming (ADP) method to solve large-scale Markov decision programming (MDP) problem. In comparison with many classic ADP methods which have large number of constraints, we formulate an alternating ADP (AADP) which have both small number of constraints and small number of variables by approximating the decision variables (instead of the objective functions in classic ADP) and write the dual of the exact LP. Also, to get the basis functions, we use kernel approximation instead of empirical choice of basis functions, which can efficiently learn nonlinear functions while retaining the expressive power. By treating option pricing as an large-scale MDP problem, we apply the AADP method to give an empirical proof that American call option will not be exercised earlier if the underlying stock has no dividend payment, which is a classic result proved by Black-Scholes model. We also make comparison of pricing options in high-dimensional with some benchmark option pricing papers which use the classic ADP to give upper and lower bound of the option price.