Note on the (non-)smoothness of discrete time value functions

TL;DR

This paper investigates the smoothness properties of the value function in discrete time stopping problems, revealing non-smoothness in both the interior and boundary of the continuation set, with implications for the Chow-Robbins game.

Contribution

It demonstrates that the value function can be non-differentiable inside the continuation set, extending known results about boundary non-smoothness, and provides examples with the Chow-Robbins game.

Findings

Value function is not differentiable on a dense subset of the continuation set.

The continuation set may be non-convex and its boundary non-smooth.

Both interior and boundary non-smoothness are demonstrated in examples.

Abstract

We consider the discrete time stopping problem \[ V(t,x) = \sup_{\tau}E_{(t,x)}[g(\tau, X_\tau)],\] where $X$ is a random walk. It is well known that the value function $V$ is in general not smooth on the boundary of the continuation set $\partial C$. We show that under some conditions $V$ is not smooth in the interior of $C$ either. More precisely we show that $V$ is not differentiable in the $x$ component on a dense subset of $C$. As an example we consider the Chow-Robbins game. We give evidence that as well $\partial C$ is not smooth and that $C$ is not convex, even if $g(t,\cdot)$ is for every $t$.