Accelerated Share Repurchase and other buyback programs: what neural networks can bring

TL;DR

This paper introduces a machine learning approach to optimize complex share repurchase contracts, overcoming limitations of traditional methods and enabling the management of higher-dimensional problems.

Contribution

It presents a novel neural network-based method for managing buyback contracts, addressing high-dimensional challenges and extending beyond classical PDE and tree techniques.

Findings

Neural network approach recovers strategies similar to traditional methods.

Method handles higher-dimensional problems without curse of dimensionality.

Enables management of complex buyback contracts previously intractable.

Abstract

When firms want to buy back their own shares, they have a choice between several alternatives. If they often carry out open market repurchase, they also increasingly rely on banks through complex buyback contracts involving option components, e.g. accelerated share repurchase contracts, VWAP-minus profit-sharing contracts, etc. The entanglement between the execution problem and the option hedging problem makes the management of these contracts a difficult task that should not boil down to simple Greek-based risk hedging, contrary to what happens with classical books of options. In this paper, we propose a machine learning method to optimally manage several types of buyback contract. In particular, we recover strategies similar to those obtained in the literature with partial differential equation and recombinant tree methods and show that our new method, which does not suffer from the curse of dimensionality, enables to address types of contract that could not be addressed with grid or tree methods.

Equations

$$A_n=\frac{1}{n}\sum _{k=1}^n{S}_k.$$

$$\{\begin{array}{l}{q}_0=0\\ q_{n+1}=q_n+v_n\delta t.\end{array}$$

$$\underset{\rho \to +\infty }{\lim }\frac{L(\rho )}{\rho }=+\infty .$$

$$\{\begin{array}{l}{X}_0=0\\ X_{n+1}=X_n+v_n{S}_{n+1}\delta t+g\left(\frac{v_n}{V_{n+1}}\right)v_n\delta t=X_n+v_n{S}_{n+1}\delta t+L\left(\frac{v_n}{V_{n+1}}\right)V_{n+1}\delta t,\end{array}$$

$${\text{PnL}}_n^Q=Q{A}_n-X_n-(Q-q_n)S_n-\ell (Q-q_n).$$

$${\text{PnL}}_n^F=F-X_n-\left(\frac{F}{A_n}-q_n\right)S_n-\ell \left(\frac{F}{A_n}-q_n\right).$$

$${\text{PnL}}_n^S=F-X_n+\alpha (q(A_n-\kappa S_0)-F{)}_{+},$$

$${\text{PnL}}_n^S=-\ell (F-X_n)+\alpha (q_n(A_n-\kappa S_0)-F{)}_{+}-\beta (q_n(A_n-\kappa S_0)-F{)}_{-},$$

$$\mathcal{A}=$$

$$\text{and }{n}^{\ast }\text{ is a }\mathcal{F}\text{-stopping time taking values in }\mathcal{N}\cup \{N\}\}.$$

$$\underset{(v,n^{\ast })\in \mathcal{A}}{\sup }\mathbb{E}[-\exp (-\gamma {\text{PnL}}_{n^{\ast }})]$$

$$\underset{(v,n^{\ast })\in \mathcal{A}}{\sup }\mathbb{E}[-\exp (-\gamma {\text{PnL}}_{n^{\ast }})]\ne -\infty$$

$$\mathrm{PnL}=\sum _{n=1}^N\prod _{k=1}^{n-1}(1-{\hat{p}}_k){\hat{p}}_n{\mathrm{PnL}}_n.$$

$$v_{\theta }(n,S,A,X,q)=Q\cdot \min \left(\left(1+{\tilde{v}}_{\theta }\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_0}-1,\frac{A-S}{S_0},\frac{q}{Q}-\frac{1}{2}\right)\right)\cdot \frac{n+1}{N},1\right)-q,$$

$$p_{\varphi }(n,S,A,X,q)=1_{n\in \mathcal{N}}\cdot \mathcal{S}\left({\nu }_{\varphi }\cdot \left(\frac{q}{Q}-{\tilde{p}}_{\varphi }\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_0}-1,\frac{A-S}{S_0}\right)\right)\right)+1_{n=N},$$

$$\mathcal{S}:x\mapsto \min \left(\max \left(\frac{2}{1+e^{-x}}-\frac{1}{2},0\right),1\right).$$

$$v_{\theta }(n,S,A,X,q)=\frac{F}{A}\cdot \frac{n+1}{N}\left(1+{\tilde{v}}_{\theta }\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_0}-1,\frac{A-S}{S_0},\frac{qA}{F}-\frac{1}{2}\right)\right)-q,$$

$$p_{\varphi }(n,S,A,X,q)=1_{n\in \mathcal{N}}\cdot \mathcal{S}\left({\nu }_{\varphi }\cdot \left(\frac{qA}{F}-{\tilde{p}}_{\varphi }\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_0}-1,\frac{A-S}{S_0}\right)\right)\right)+1_{n=N},$$

$$v_{\theta }(n,S,A,X,q)=1_{X<F}\cdot \frac{F-X}{S}\max \left(\min \left(\frac{1}{N-n}\left(1+{\tilde{v}}_{\theta }\left(\cdots \right)\right),1\right),0\right),$$

$$p_{\varphi }(n,S,A,X,q)=1_{n\in \mathcal{N}}\cdot \mathcal{S}\left({\nu }_{\varphi }\cdot \left(\frac{X}{F}-{\tilde{p}}_{\varphi }\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_0}-1,\frac{A-S}{S_0},\frac{q{S}_0}{F}-\frac{1}{2}\right)\right)\right)+1_{n=N},$$

$$-\frac{1}{\gamma }\log \mathbb{E}\left[\exp (-\gamma \text{PnL})\right]$$

$$\mathbb{E}\left[\mathrm{PnL}\right]$$

$$\mathbb{E}\left[\prod _{k=1}^{n-1}\left(1-{\hat{p}}_k\right){\hat{p}}_n|{\mathcal{F}}_N\right]=\mathbb{E}\left[\prod _{k=1}^{n-1}\left(1-1_{{\widetilde{ϵ}}_k<p_k}\right)1_{{\widetilde{ϵ}}_n<p_n}|{\mathcal{F}}_N\right]=\prod _{k=1}^{n-1}\left(1-p_k\right)p_n,$$

$$\mathbb{E}\left[{\mathrm{PnL}}^2\right]=\mathbb{E}\left[\sum _{n=1}^N\prod _{k=1}^{n-1}\left(1-p_k\right)p_n{\mathrm{PnL}}_n^2\right].$$

$$-\frac{1}{\gamma }\log \mathbb{E}\left[\exp (-\gamma \text{PnL})\right]$$

$$\forall (x_1,\dots ,x_n)\in [0,1{]}^n,f(x_1,\dots ,x_n)=\sum _{q=1}^{2n+1}{\chi }_q\left(\sum _{p=1}^n{\varphi }_{p,q}(x_p)\right).$$

$${\Psi }^l:x\in {\mathbb{R}}^{d_{l-1}}\mapsto g^l(A^l{\Psi }^{l-1}(x)+b^l),\forall l\in \{1,\dots ,L\},$$

$$\left\{x\in K\mapsto \sum _{j=1}^{d_1}{A}_{1,j}^{2}h\left(\sum _{k=1}^{d_0}{A}_{j,k}^1{x}_k+b_k^1\right)|d_1\in {\mathbb{N}}^{\ast },A^1\in {\mathbb{R}}^{d_1\times d_0},b^1\in {\mathbb{R}}^{d_1},A^2\in {\mathbb{R}}^{1\times d_1}\right\}$$

Full text

Key words: ASR contracts, Optimal stopping, Stochastic optimal control, Deep learning, Recurrent neural networks, Reinforcement learning.

1 Introduction

Payout policy has been a major research topic in corporate finance since the payout irrelevance proposition of Modigliani and Miller [32] stating the equivalence of dividend payment and share buyback in an idealised market without taxes, frictions, and information asymmetries. When taxes, frictions, and information asymmetries enter the scene, there could be reasons to prefer share buybacks over dividend payments, or vice versa. In practice, in addition to fiscal motives in some regions, share buybacks are often favoured for signalling stock price undervaluation, for deterring takeover, or for offsetting the dilution effect associated with stock options (see [2, 15] for a review on payout policy).

Share buybacks can be carried out using several methods. Until the end of the 80s, share repurchases were predominantly made via fixed-price tender offers and Dutch auctions.444Privately negotiated repurchases also existed and continue to exist. Then, in the 90s, open market repurchases (OMRs) took over and represented the vast majority of share buyback programs (see [35]). However, as reported for instance in [8], after a share repurchase announcement, a substantial number of companies usually do not commit to it. In order to make a credible commitment, an increasing number of firms started, from the early 2000s, to sign contracts with investment banks to delegate buyback programs in the form of VWAP-minus programs. The main examples of such contracts are Accelerated Share Repurchase (ASR) contracts.

In a nutshell, ASR contracts work as follows. Upon signature of an ASR contract between a firm and an investment bank, the latter delivers shares to the former by borrowing them from shareholders (typically institutional investors). Subsequently the bank has a short position and needs to buy shares in the open market to return them back to the lenders. The contract typically involves an option component to determine either the price per share paid by the firm, the number of shares it receives, or both. This option component is usually of the Asian type with Bermudan exercise features, or even more complex in the case of profit-sharing programs (see Section 2 for more details).

In addition to higher credibility (see [8]), the motives of firms carrying out buyback through accelerated programs are numerous. An important segment of the academic literature deals with the financial reporting advantages and the immediate boost of earnings per share (EPS) provided by ASR contracts. For instance, [29] and [30] find evidence of EPS enhancement as a motive of ASR adoption,555The literature discusses for instance the incentive of management to sign ASR contracts to boost EPS for increasing performance-based compensation, see [30]. but this finding has to be put in perspective because of other studies such as [1, 8, 12, 25] finding little evidence. The literature also discusses the signalling content of ASR over OMR programs, as the commitment associated with ASRs reinforces the classical undervaluation signal of share buyback programs (see [12, 25, 29]).

The economic literature on ASRs also deals with the short- and long-term effects of ASR announcement on the firm stock price. Many papers suggest indeed an immediate increase in the stock price, although the amplitude of this effect is debated (see for instance [1, 6, 25, 31, 36]). Some also discuss price manipulations of firms willing to reduce the price of stocks before the announcement of ASR programs (see [13, 14]). Market microstructure changes around ASR announcements are also discussed in [28].

In spite of an extensive economic literature on ASR contracts, the pricing and management of complex buyback contracts has rarely been tackled. Pioneer works on the subject include that of Jaimungal et al. [24] and papers by Guéant et al. [19, 20]. They all show that ASR contracts should not be managed like traditional equity derivatives, i.e. managed with Greeks, because the execution problem at the heart of these contracts cannot be disentangled from the option component. The payoff of the option constitutes indeed, in most cases, a partial hedge for the execution process. Moreover, the volumes to be executed are often very large and execution costs must be taken into account. Furthermore, there are often participation constraints in buyback programs preventing to buy more than a given proportion of the daily volume, or even forbidding the use of stock selling.

In [24], the authors focus on ASR contracts with fixed number of shares and American exercise. They propose a continuous-time model where the stock price is modeled as a geometric Brownian motion with a drift reflecting permanent market impact, and add quadratic execution costs as in Almgren-Chriss models (see [3, 4]). The strategy they propose is optimal for a bank maximising its expected profit and loss (PnL) and penalising inventory (a penalty that can also be regarded as a form of ambiguity aversion as far as the stock price process is concerned). Jaimungal et al. derive the dynamic programming equation associated with the problem: a degenerate quasi-variational inequality in dimension 4 (time + 3 state variables). Interestingly, in an attempt to tame the degeneracy of the equation, they introduce the ratio between the stock price and its average value since inception and subsequently reduce the dimensionality of the problem. In addition to obtaining a new quasi-variational inequality – this time in dimension 3, including time –, they show that the exercise boundary only depends on the time to maturity and the above ratio.

The case of ASR contracts with fixed number of shares is also dealt with in the paper [20] by Guéant, Pu, and Royer who proposed a discrete-time model with a general execution cost function, and an expected utility objective function. As in [24], they show that the problem boils down to a set of equations with 3 variables; here time to maturity, the number of shares to be bought, and the difference between the current stock price and the average price since inception (and not the ratio because of the different assumptions regarding price dynamics). The case of ASR contracts with fixed notional is dealt with in [19] and it must be noted that there is no similar dimensionality reduction in that case. It is noteworthy that more complex VWAP-minus programs, such as profit-sharing programs, are not dealt with in the literature.

Because of its high-dimensional nature, it is natural to try solving the problem of pricing and managing ASRs and other (more complex) VWAP-minus programs with the help of neural networks instead of grids or trees as in the above literature. This paper proposes a machine learning approach involving recurrent neural networks to find the optimal execution strategy associated with different types of VWAP-minus programs: ASRs with fixed number of shares, ASRs with fixed notional, and profit-sharing contracts.

In recent years, following the craze regarding neural networks, several research papers have encouraged the idea that neural network techniques could be a way to tackle financial issues suffering from the curse of dimensionality. In particular, several papers written by Jentzen and collaborators – see for instance [9, 21, 34] – proposed new methods, based on neural networks, to approximate the solutions of linear and nonlinear parabolic partial differential equations (PDE). In particular, [34] solves linear PDEs including that of Black and Scholes with correlated noises and that of Heston, and [21] solves the nonlinear equation associated with the Black-Scholes model when different interest rates are considered for borrowing and lending.666Interestingly, these papers do not approximate directly the solution of the PDEs, but their (space-)gradient (related to the actions in the vocabulary of reinforcement learning). In other words, prices must be deduced from Greeks and not the other way round as with the classical tools of mathematical finance. A group of researchers around Pham (see [7, 23]) recently proposed other methods based on neural networks to solve optimal control problems with applications to energy issues, and proved results of convergence. In finance, papers on the hedging of options with (deep) neural network techniques include the famous “Deep Hedging” paper (see [11]) written by Buehler et al. that uses a semi-recurrent neural network. The case of American and Bermudan payoffs is also addressed in [10] with an interesting idea that we also use, though in a slightly different manner: the relaxation of the optimal stopping decision.

Our approach is innovative in that, in addition to looking for the best execution strategy using a recurrent neural network, we do not look directly for the optimal stopping time, but rather for the optimal probability to stop at each step, given the current state. This relaxation allows to go from a discrete decision problem to a continuous one, and therefore enables the use of gradient descent tools. In practice, we use a second neural network for modelling the probability to stop. Our approach recovers results similar to those of [19, 20] in the case of ASR contracts. Compared to the approaches based on the dynamic programming principle, our approach has a number of advantages: it does not require one to solve non-linear PDEs in a high-dimensional space, and thus allows to handle more sophisticated contracts – see our treatment of VWAP-minus profit-sharing contracts – and allows essentially any price dynamics unlike what happens with the grid or tree approaches developed in the literature.

In Section 2, we describe the three different types of buyback contracts addressed in the paper: two types of ASR contracts and one VWAP-minus profit-sharing contract. In Section 3, we propose a discrete-time model similar to that of [18, 19, 20] and define the objective functions. In Section 3, we also describe the architecture of our deep recurrent neural network to approximate the optimal strategy for managing the different contracts. In Section 4, we provide numerical results and discuss our findings. An appendix is dedicated to neural networks in order to provide the readers with ideas that are still seldom used in mathematical finance.

2 Buyback contracts

In this paper we consider three different types of buyback contract: the two types of ASR contract tackled in [18, 19, 20, 24], and one VWAP-minus profit-sharing contract never addressed in the academic literature. The termsheets of these contracts can be summarised as follows:

1. I.

ASR contract with fixed number of shares:

At time $t=0$, the bank borrows $Q$ shares from the firm’s shareholders (usually institutional investors) and delivers these shares to the firm in exchange for the current Mark-to-Market (MtM) value of these assets ($QS_{0}$).777Here we consider the case of a pre-paid ASR contract. The case of a post-paid ASR contract is the same, if funding and interest rate are ignored. 2. 2)

The bank has to progressively buy $Q$ shares in the open market to give them back to the initial shareholders and return to a flat position on the stock. 3. 3)

The final settlement of the contract is associated either with the early exercise of an option or with the expiry of the contract (at time $T$). If the bank decides to early exercise the option at time $\tau\in\mathcal{T}$, where $\mathcal{T}\subset(0,T)$ is the set of possible early exercise dates specified in the contract, then the firm pays to the bank the difference between the average market price between [math] and $\tau$ (in this section, we denote by $A_{t}$ the average price between [math] and $t$) and the price at inception $S_{0}$. This can be regarded as the bank being long a Bermudan option with Asian payoff $Q(A_{\tau}-S_{0})$. If the contract goes to expiry the final payoff is instead $Q(A_{T}-S_{0})$. 2. II.

ASR contract with fixed notional:

At time $t=0$, the firm pays to the bank a fixed amount of cash $F$. In return, the bank delivers to the firm $Q$ shares borrowed from the firm’s shareholders, where $Q=\zeta\frac{F}{S_{0}}$ ($\zeta$ is usually around $80\%$). 2. 2)

The bank has to progressively buy back $Q$ shares in the open market to give them back to the initial shareholders. 3. 3)

The final settlement of the contract is associated either with the early exercise of an option or with the expiry of the contract (at time $T$). If the bank decides to early exercise the option at time $\tau\in\mathcal{T}$, where $\mathcal{T}\subset(0,T)$ is the set of possible early exercise dates specified in the contract, then there is a transfer of $\frac{F}{A_{\tau}}-Q$ shares from the bank to the firm, so that the actual number of shares acquired by the firm is $\frac{F}{A_{\tau}}$. If the contract goes to expiry, then there is a transfer of $\frac{F}{A_{T}}-Q$ shares from the bank to the firm.

Remark 1**.**

*In practice, for both types of ASR, there is often a discount proposed to the firm: the bank gives back part of the option value in the form of a discount on the average price – hence the expression VWAP-minus used for most of these programs. Considering this discount does not raise any difficulty when using our approach, unlike what would happen with classical methods.888For instance, the dimensionality reduction obtained through a change of variables in [20] does not work anymore in presence of a multiplicative discount.

3. III.

VWAP-minus profit-sharing contract:

At time $t=0$, there is no initial transaction. 2. 2)

The bank has to buy shares in the open market on behalf of the client either until an amount of cash equal to $F$ has been spent or until the expiry of the contract (at time $T$). For this type of contract, selling is prohibited. 3. 3)

If the contract expires before the required amount of cash is spent, the contract is settled by the payment of a penalty by the bank to the firm.999This should never happen as $T$ is chosen to ensure the possibility of the delivery. Otherwise, once an amount of cash equal to $F$ has been spent (we denote by time $\tau_{0}$ the occurrence of that event), the bank becomes long a Bermudan option with expiry date $T$ and payoff $\alpha(q(A_{\tau}-\kappa S_{0})-F)_{+}$, where:

• •

$q$ is the number of shares bought by the bank on behalf of the firm against the amount $F$;

• •

$\tau\in\mathcal{T}\cap[\tau_{0},T]$ designates a stopping time (as in all Bermudan/American options), where $\mathcal{T}\subset(0,T]$ is the set of possible exercise dates specified in the contract;

• •

$\alpha$ is the proportion of profit sharing (typically $25\%$);

• •

$\kappa$ is a hurdle rate required by the firm (typically below $1\%$).

In other words, the bank is incentivised to carry out the execution at a better price than the average price minus a discount.

Remark 2**.**

*These contracts are common in the brokerage and corporate derivatives industry, but it is not clear that they really give the bank an incentive to carry out a good execution in all situations. If, indeed, the beginning of the execution process is poor, and if the bank subsequently realises that the option will be worth almost nothing, then it has no reason to provide the best possible execution to the client. For this reason banks, in order to give the best service to the client, should manage the option as if the payoff was $\alpha(q(A_{\tau}-\kappa S_{0})-F)$ or $\alpha(q(A_{\tau}-\kappa S_{0})-F)_{+}-\beta(q(A_{\tau}-\kappa S_{0})-F)_{-}$ (where $\beta\in[0,\alpha)$) instead of $\alpha(q(A_{\tau}-\kappa S_{0})-F)_{+}$.

3 The model

3.1 Mathematical setting

3.1.1 Dynamics of the state variables

We consider a discrete-time model where each period of time of length $\delta t$ corresponds to one day. In other words, given a contract with maturity date $T$ corresponding to $N$ days ($T=N\delta t$), we consider the subdivision $(t_{n}=n\delta t)_{0\leq n\leq N}$ of the interval $[0,T]$. We denote by $\mathcal{N}=\{n\in\{0,\ldots,N\}|t_{n}\in\mathcal{T}\}$ the set of indices corresponding to the possible (early) exercise dates.

We consider a probability space $(\Omega,\mathbb{P})$ and a listed firm whose stock price is modelled by a stochastic process $(S_{n})_{n}$. We denote by $(\mathcal{F}_{n})_{n}$ the completed filtration generated by $(S_{n})_{n}$ (i.e. we assume that $\mathcal{F}_{0}$ contains all the $\mathbb{P}$-null sets).

Remark 3**.**

It is noteworthy that we do not set a particular model for the price dynamics.

For $n\in\{1,\ldots,N\}$, the running average price of the stock over $\{t_{1},\ldots,t_{n}\}$ is denoted by

[TABLE]

Let us consider a bank in charge of buying shares of that firm. We assume that the bank executes an order each day, and we denote by $(v_{n}\delta t)_{n}$ the daily volumes of transactions: $v_{0}\delta t$ for the first day, $v_{1}\delta t$ for the second day, etc. Subsequently, the number of shares $(q_{n})_{n}$ bought by the bank in the market is given by

[TABLE]

For each share bought over the $n$-th day the bank pays $S_{n}+g\left(\frac{v_{n}}{V_{n+1}}\right)$, where $g$ is a nonnegative function modelling execution costs per share and $(V_{n})_{n}$ is the market volume process, assumed to be deterministic. In other words, the trader pays the reference price for the $n$-th day plus execution costs depending on her participation rate to the market over the $n$-th day.

Following [18], we consider the function $L:\rho\in\mathbb{R}\mapsto\rho g(\rho)$ and assume that $g$ is such that $L$ verifies the following assumptions:

• •

$L$ is strictly convex on $\mathbb{R}$, increasing on $\mathbb{R}_{+}$, and decreasing on $\mathbb{R}_{-}$;

• •

$L$ is asymptotically superlinear, i.e.:

[TABLE]

The resulting cumulative cash spent by the bank modelled by $(X_{n})_{n}$ has the following dynamics:

[TABLE]

In the following, we first compute the profit and loss associated with each type of contract. Then, we introduce the set of admissible controls and propose an objective function that could be used by the bank to carry out optimisation.

3.1.2 Profit and Loss

1. I.

ASR contract with fixed number of shares:

No matter if the bank chooses to early exercise on day $n\in\mathcal{N}$ or if the contract expires on day $n=N$, the bank has to acquire $Q-q_{n}$ shares. We assume that these remaining shares could be purchased at price $S_{n}$ plus execution costs. The resulting amount of cash spent by the bank at time $n$ is $(Q-q_{n})S_{n}+\ell(Q-q_{n})$, where $\ell:\mathbb{R}\mapsto\mathbb{R}_{+}$ satisfies the same properties as the execution cost function $L$.

At exercise date or at expiry (day $n$) the bank receives from the firm an amount of cash equal to $QA_{n}$. The resulting profit and loss of the bank is

[TABLE] 2. II.

ASR contract with fixed notional:

No matter if the bank chooses to early exercise on day $n\in\mathcal{N}$ or if the contract expires on day $n=N$, the bank has to acquire $\frac{F}{A_{n}}-q_{n}$ shares. We assume that these remaining shares could be purchased at price $S_{n}$ plus execution costs. The resulting amount of cash spent by the bank at time $n$ is $\left(\frac{F}{A_{n}}-q_{n}\right)S_{n}+\ell\left(\frac{F}{A_{n}}-q_{n}\right)$, where $\ell:\mathbb{R}\mapsto\mathbb{R}_{+}$ is as above.

At exercise date or at expiry (day $n$) the bank receives from the firm an amount of cash equal to $F$. The resulting profit and loss of the bank is

[TABLE] 3. III.

VWAP-minus profit-sharing contract:

If the bank manages to spend the amount $F$ before expiry, then its profit and loss is

[TABLE]

where $n$ corresponds to the date of exercise of the option.101010In practice, $X_{n}$ should be equal to $F$. Otherwise, we assume that the profit and loss at expiry date is just a penalty.

In our approach, we consider (i) that the option can be exercised even if the amount $F$ has not been spent and (ii) that once an amount of cash $F$ has been spent the bank stops trading. Moreover, we consider the modification of the profit and loss discussed in Remark 2. This results in the following modified profit and loss formula:

[TABLE]

where $\ell:\mathbb{R}\mapsto\mathbb{R}_{+}$ is as above.

If the bank exercises the option before the amount $F$ has been spent, then the penalty associated with $\ell$ is paid and we assume that it is large enough to compensate the profit sharing term (should it be positive) if $X_{n}$ is far below $F$. Otherwise, the payoff is just the same as above, except when it comes to the additional $\beta$ term.

3.1.3 Objective function

Before introducing the objective function let us first define the set of admissible controls. We consider minimal and maximal market participation rates. In other words, we impose the market participation constraints $\underline{\rho}V_{n+1}\leq v_{n}\leq\overline{\rho}V_{n+1}$, where $\overline{\rho}$ is positive and $\underline{\rho}$ can be of either sign.111111Constraints of this type are sometimes specified explicitly in the contract.

Therefore the set of admissible strategies of the bank can be represented as follows:

[TABLE]

To be consistent with [19, 20], we consider that the bank is willing to maximise the expected CARA utility of its PnL. Therefore, the optimisation problem faced by the bank is the following:

[TABLE]

where $\gamma$ is the risk aversion parameter of the bank and PnL is either $\textrm{PnL}^{Q}$, $\textrm{PnL}^{F}$ or $\textrm{PnL}^{S}$.

Remark 4**.**

We assume that the dynamics of the stock is chosen so that the above problem has a solution, i.e.

[TABLE]

3.2 Relaxation and mean-variance approximation: towards a machine learning approach

3.2.1 Relaxation of the optimal stopping problem

Given the structure of the problem, the optimal number of shares to be bought on day $n+1$ can be written as a closed-loop control $v(n,S_{n},A_{n},X_{n},q_{n})$. Similarly, the optimal decision to exercise the option can be written as: $\mathbf{1}_{\{n^{*}=n\}}=p(n,S_{n},A_{n},X_{n},q_{n})$.

Since the function $p$ takes values in $\{0,1\}$, this problem is not suitable for the optimisation methods commonly associated with neural networks, e.g. stochastic gradient descent. In this regard, we extend the set of admissible controls to allow stochastic stopping decisions.

More precisely, an admissible strategy is determined by:

• •

the number of shares to be bought on each day, modelled (up to the $\delta t$ multiplicative term) by a $\mathcal{F}$-adapted process $(v_{n})_{n}$;

• •

the stochastic stopping policy $(p_{n})_{n}$, which is a $\mathcal{F}$-adapted process that takes values in the interval $[0,1]$ with $p_{n}=\mathbf{1}_{n=N}$ if $n\notin\mathcal{N}$.

In order to sample effective stopping decisions based on the stochastic stopping policy $(p_{n})_{n}$, we introduce an extended $\sigma$-algebra $\mathcal{G}\supset\mathcal{F}_{N}$ and i.i.d random variables $(\tilde{\epsilon}_{n})_{n}$ defined on $(\Omega,\mathbb{P},\mathcal{G})$, uniform on $[0,1]$, and assumed to be independent of $\mathcal{F}_{N}$.

The effective stopping time $n^{\star}$ is then defined as $\min\left\{n\in\mathcal{N}\cup\{N\}|\tilde{\epsilon}_{n}\leq p_{n}\right\}$, so that the stopping decision $\widehat{p}_{n}$ defined by $\widehat{p}_{n}=\mathbf{1}_{\tilde{\epsilon}_{n}<p_{n}}$ is conditionally distributed as a Bernoulli with parameter $p_{n}$ given $\mathcal{F}_{n}$.

Therefore the PnL of the strategy is given by:

[TABLE]

We search for the optimal strategy $v$ in the form of $v_{\theta}(n,S_{n},A_{n},X_{n},q_{n})$ for $n\in\{0,\ldots,N-1\}$, and $p$ in the form of $p_{n}=p_{\phi}(n,S_{n},A_{n},X_{n},q_{n})$ for $n\in\mathcal{N}$, both of them lying in a finite-dimensional set of functions parameterised by $\theta$ and $\phi$ respectively.

Remark 5**.**

For the fixed number of shares and fixed notional ASR contracts, the optimal strategy does not depend on the cash variable when using a CARA utility framework (see [19, 20]). Therefore, the cash variable is absent of $v_{\theta}$ and $p_{\phi}$ in these cases.

In our relaxed setting, the objective function has then a differentiable dependency on the parameters of the neural networks.121212A similar idea is used in [10] to handle American options with neural networks.

3.2.2 Neural networks

For the neural networks to be robust with respect to scaling effects, we ensure that the variables that are the inputs of the neural networks are dimensionless and centered. Using the finding of [19, 20], we give as an input $\frac{A-S}{S_{0}}$ instead of $A$, as the strategy has a strong dependency on the spread between the spot price and the running average. Likewise, the outputs of the networks are designed as perturbations of naive strategies (see below for details).

1. I.

ASR contract with fixed number of shares:

We parameterise the rate of share repurchase $v_{\theta}$ by:

[TABLE]

where $\tilde{v}_{\theta}$ is a neural network consisting of 4 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output.

It is noteworthy that if $\tilde{v}_{\theta}$ is equal to [math], then the portfolio to reach at step $n+1$ is $\frac{n+1}{N}Q$, which corresponds to the trading schedule of a trader buying the same amount of shares each day until maturity.

The stochastic stopping policy $p_{\phi}$ is represented by:

[TABLE]

where $\tilde{p}_{\phi}$ is a neural network consisting of 3 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output, $\nu_{\phi}$ is a scaling parameter, and $\mathcal{S}$ is the activation function defined by:

[TABLE]

We use the activation funcion $\mathcal{S}$ that is a modified version of the logistic function (rescaled and bounded to $[0,1]$) to allow the values 0 and 1 to be reached.

The output of the network $p_{\phi}$ can be interpreted as the frontier in terms of the ratio $\frac{q}{Q}$, depending on $n$, $S$ and $A$, above which we exercise the option. 2. II.

ASR contract with fixed notional:

We parameterise the rate of share repurchase $v_{\theta}$ by:

[TABLE]

where $\tilde{v}_{\theta}$ is a neural network consisting of 4 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output.

It is noteworthy that if $\tilde{v}_{\theta}$ is equal to [math], then the portfolio to reach at step $n+1$ is $\frac{n+1}{N}\frac{F}{A}$, which corresponds to a natural naive trading schedule.

The stochastic stopping policy $p_{\phi}$ is represented by:

[TABLE]

where $\tilde{p}_{\phi}$ is a neural network consisting of 3 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output, and $\nu_{\phi}$ is a scaling parameter.

The output of the network $p_{\phi}$ here can be interpreted as the frontier in terms of the ratio $\frac{qA}{F}$, depending on $n$, $S$ and $A$, above which we exercise the option. 3. III.

VWAP-minus profit-sharing contract:

We parameterise the rate of share repurchase $v_{\theta}$ by:

[TABLE]

where $\tilde{v}_{\theta}\left(\cdots\right)$ stands for $\tilde{v}_{\theta}\left(\frac{n}{N}-\frac{1}{2},\frac{S}{S_{0}}-1,\frac{A-S}{S_{0}},\frac{X}{F}-\frac{1}{2},\frac{qS_{0}}{F}-\frac{1}{2}\right)$, and $\tilde{v}_{\theta}$ is a neural network consisting of 5 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output.

It is noteworthy that if $\tilde{v}_{\theta}$ is equal to [math], then the cash spent is equal (up to the execution costs) to the ratio of the remaining cash to spend to the remaining number of days until maturity, which is a natural naive strategy. The max and min functions prevent us from selling and over-buying.

The stochastic stopping policy $p_{\phi}$ is represented by:

[TABLE]

where $\tilde{p}_{\phi}$ is a neural network consisting of 4 inputs, a hidden layer of 50 neurons with ReLU activation function and 1 output, and $\nu_{\phi}$ is a scaling parameter.

The output of the network $p_{\phi}$ here can be interpreted as the frontier in terms of the ratio $\frac{X}{F}$, depending on $n$, $S$ and $A$, above which we exercise the option.

3.2.3 Objective function approximation

We could use a stochastic gradient descent or a mini-batched gradient descent on the expected CARA utility objective function to approximate an optimal trading and an optimal stopping strategy. However the very fact that the utility is exponential typically causes numerical issues. For that reason, we consider the classical Arrow-Pratt (see for instance [33]) approximation of the expected CARA utility objective function by a mean-variance objective function:131313It is important to note that we could have chosen a mean-variance objective function from the very beginning. The reason why we started with an exponential utility is to relate our paper to [19, 20].

It is also important to recall that the mean-variance approximation of the certainty equivalent associated with a CARA utility function (i) turns out to be exact in the case of Gaussian risks and (ii) corresponds to a first order Taylor expansion in the risk aversion parameter (around 0).

[TABLE]

In our relaxed setting, we have

[TABLE]

since

[TABLE]

where we used the fact that $(p_{1},\ldots,p_{n})$ is $\mathcal{F}_{N}$-measurable, and that $\tilde{\epsilon}_{1},\ldots,\tilde{\epsilon}_{n}$ are i.i.d. and independent of $\mathcal{F}_{N}$.

Similarly,

[TABLE]

Subsequently,

[TABLE]

Therefore, using a Monte-Carlo approximation with $I$ trajectories of prices $(S_{n}^{i})_{0\leq n\leq N,1\leq i\leq I}$, and the resulting stopping policy $(p_{n}^{i})_{1\leq n\leq N,1\leq i\leq I}$ and profit and losses $(\textrm{PnL}_{n}^{i})_{1\leq n\leq N,1\leq i\leq I}$, we can consider the following approximation

[TABLE]

Given the sampled trajectories $(S_{n}^{i})_{0\leq n\leq N,1\leq i\leq I}$, the right-hand side of the above equation depends only on $\theta$ and $\phi$. Therefore using automatic differentiation tools we can perform gradient descent on this proxy of the objective function.

4 Numerical results

In this section we illustrate the practical use of our method. We consider the reference case described below which corresponds to rounded values for the stock Total SA, deliberately chosen to be the same as in [20] in order to show that the strategies obtained with our method are similar to those derived in [20] for an ASR contract with fixed number of shares.

For the same comparison purpose, we train the neural networks with arithmetic Brownian motion price trajectories $S_{n+1}=S_{n}+\sigma\sqrt{\delta t}\epsilon_{n+1}$, where $(\epsilon_{n})_{n}$ are i.i.d. $\mathcal{N}(0,1)$ random variables. Contrary to what happens with the method presented in [20], our method can be used with almost any price dynamics or even historical data.141414Unfortunately, historical time series are often not long enough. A new practice consists in using data simulated with generative models calibrated to historical data (e.g. generative adversarial networks – see [17]).

More precisely, we consider the following market model:

• •

$S_{0}=45$ €;

• •

$\sigma=0.6$ €$\cdot\text{day}^{-1/2}$, corresponding to an annual volatility approximately equal to $21\%$;

• •

$T=63$ trading days. The set of possible early exercise dates is $\mathcal{N}=[22,62]\cap\mathbb{N}$;

• •

$\forall n\in\{1,\ldots,N\},V_{n}=V=4\ 000\ 000$ shares$\cdot$ $\text{day}^{-1}$;

• •

$L(\rho)=\eta|\rho|^{1+\phi}$ with $\eta=0.1$ € $\cdot\mbox{share}^{-1}\cdot\text{day}^{-1}$ and $\phi=0.75$.

4.1 ASR contract with fixed number of shares

For this contract, we consider the following characteristics:

• •

$Q=20\ 000\ 000$ shares;

• •

$\ell:q\mapsto Cq^{2}$ for the terminal penalty, where $C=2\cdot 10^{-7}$ € $\cdot\mbox{share}^{-2}$;

• •

$\underline{\rho}=-\infty,\overline{\rho}=+\infty$, meaning that there is no participation constraints.

Our choice for the risk aversion parameter is $\gamma=2.5\cdot 10^{-7}$ €-1.

Let us consider three different trajectories for the price in order to exhibit several features of the optimal strategy of the bank.

The first price trajectory exhibits an upward trend. In this case, the optimal strategy of the bank consists in buying the shares slowly to minimise execution costs, as illustrated in Figure 1.

The second price trajectory exhibits a downward trend. In that case, the bank has an incentive to exercise rapidly, even without all the required shares being bought (see Figure 2). Indeed, as the price stays below its average, the latter is pulled down over time, making it less profitable to postpone the exercise of the option.

The third price trajectory we consider corresponds to the price decreasing and then increasing. As in both preceding examples, we see in Figure 3 that the behaviour of the bank is strongly linked to the relative position of the price to its running average. At the beginning of the contract, when the price is below its running average, the bank is acquiring shares at a high pace. Afterwards, when the price goes above its running average, it is not profitable anymore for the bank to accelerate execution. Instead, the bank is incentivised to delay the exercise of the option and it is even selling shares in order to stay close to the strategy $q_{n}=\frac{n}{N}Q$ because the risk associated with that strategy is hedged by the payoff of the ASR contract.

Now, as in [20], we study the effects of the parameters on the optimal strategy. More precisely, we focus on the execution cost parameter $\eta$ and the risk aversion parameter $\gamma$.

Let us focus first on execution costs and more precisely on the liquidity parameter $\eta$. We consider our reference case with $4$ values for the parameter $\eta$: $0.01,0.1,0.2$, and $0.5$. As we can see in Figure 4, corresponding to the third price trajectory, the less liquid the stock, the smoother the optimal strategy to avoid abrupt trading and round trips.

The optimal values of the mean-variance criterion for the different values of the parameter $\eta$ are presented in the table below:

[TABLE]

As expected, the more liquid the stock, the more profitable the contract for the bank.

Let us come now to risk aversion. We consider our reference case with $4$ values for the parameter $\gamma$: $0,2.5\cdot 10^{-9},2.5\cdot 10^{-7},$ and $5.0\cdot 10^{-7}$. Figure 5 shows the influence of $\gamma$ on the optimal strategy. We see that the more risk averse the bank, the closer to the naive strategy (i.e. $q_{n}=\frac{n}{N}Q$) its strategy. This is intuitive as the risk associated with this strategy is perfectly hedged by the payoff of the ASR contract. At the other end of the spectrum, when $\gamma=0$, the corresponding strategy is much more aggressive at the beginning of the contract in order to be able to benefit from the optionality as soon as possible (because the function $\ell$ we have chosen provides a very strong incentive to have only a few shares to buy at the time of early exercise). What prevents the bank from buying instantaneously is just execution costs. An interesting point is also that, when $\gamma=0$, the optimal strategy does not involve any stock selling.151515Since neural networks are just approximations, sometimes we can see some small deviations from the optimal buy-only strategy in the no risk aversion case.

The mean-variance values for the different values of the risk aversion parameter $\gamma$ are presented in the table below:

[TABLE]

Unsurprisingly, the more risk averse the bank, the lower the optimal value of the mean-variance criterion.

As our optimisation problem is not convex, the optimisation procedure might lead to a local optimum. Because of the random initialisation of the neural networks weights and because of Monte Carlo sampling, the learning process is not always the same. Figure 6 illustrates two very different learning curves associated with two different instances of the learning procedure with $\gamma=5\cdot 10^{-7}$ €-1. We see that the optimisation process for the second instance stalls in a suboptimal state (with a mean-variance score slightly below 0), whereas the first instance manages to reach a state with a significantly higher score.

Interestingly, the suboptimal strategy associated with the second learning instance consists in buying shares at a constant pace until having the required quantity $Q$ of shares and exercising immediately the option, regardless of the price trajectory (see Figure 7). It is not surprising that this strategy could be a local optimum as the option payoff provides a perfect hedge for the execution process.

In order to deter the learner from being caught in the domain of attraction of the type of local optimum described above, we can modify the objective function by setting $\gamma$ to 0 over the first training epochs in order to remove the incentive to hedge. We refer to this procedure as pretraining.

We illustrate in Figure 8 the learning curve associated with the learning procedure where we performed pretraining over the first 100 epochs, and we compare it to the two above examples without pretraining. From this graph, we see that pretraining the network helps to avoid this type of local optimum. Moreover, when pretraining is used, we see in Figure 8 that the learning curve does not exhibit an intermediate plateau.

Remark 6**.**

It should be mentioned that our pretraining procedure enables to avoid a trivial local optimum, but does not theoretically ensure the convergence towards a global optimum (this is a common issue in machine learning, especially with neural networks). However, the strategies we obtain are in line with the results of our previous works [19, 20] based on solving Bellman equations. We believe therefore that our method succeeds in reaching a global optimum.

4.2 ASR contract with fixed notional

For this contract, we consider the following characteristics:

• •

$F=900\ 000\ 000$ €;

• •

$\ell:q\mapsto Cq^{2}$ as terminal penalty, where $C=2\cdot 10^{-7}$ € $\cdot\mbox{share}^{-2}$;

• •

$\underline{\rho}=-\infty,\overline{\rho}=+\infty$, meaning that there is no participation constraints.

We choose the risk aversion parameter $\gamma=2.5\cdot 10^{-7}$ €-1.

In Figures 9, 10 and 11, we plot the strategies obtained for the fixed notional ASR contract (for the same three price trajectories as above). The targeted number of shares is represented by a solid line (it is not constant due to the stock price change).

Now, let us compare the strategies associated with the two types of ASR contract. In Figures 12, 13 and 14, we see that we obtain similar strategies for both types of ASR: accelerating purchase when the difference between the average price and the price is positive and decelerating purchase or even selling when that difference is negative.

4.3 VWAP-minus profit-sharing contract

For this contract, we consider the following characteristics:

• •

$F=900\ 000\ 000$ €;

• •

$\alpha=25\%$;

• •

$\kappa=0.5\%$;

• •

$\ell:x\mapsto Cx^{2}$ as terminal penalty, where $C=2\cdot 10^{-9}$ €-1;

• •

$\underline{\rho}=0,\overline{\rho}=+\infty$, reflecting the prohibition to sell.

To manage the contract, we consider the modified payoff described in Remark 2 and in Section 3.1.2 with $\beta=5\%$: the bank does not only get part of the profit but also part of the loss.

We choose $\gamma=10^{-6}$ €-1 so that $\alpha\gamma=2.5\cdot 10^{-7}$.

The strategies obtained with our neural network algorithm for this type of contract are plotted in Figures 15, 16 and 17.161616It must be mentioned that, in the case of this type of contract, we used the value $C=\frac{0.01}{F}$ for the final penalty function during the pretraining phase. Here we represent our strategy in terms of the cash spent in repurchasing because cash is the crucial variable for this contract.

We see that the strategy consists in accelerating the purchase process when the price goes below its average and decelerating it when the price increases above its average. In the case of this contract, there is no round trip as selling is prohibited. This explains in particular the shape of the execution strategy in the case of the third price trajectory.

It is interesting to notice (see Figures 18, 19 and 20) that this strategy is similar to that of an ASR contract with fixed number of shares (with the same trading constraints) when one compares the proportion of the cash spent in the case of the former contract with the proportion of shares bought in the case of latter contract.

Conclusion

In this paper, we propose a machine learning approach involving recurrent neural networks to find the optimal strategy associated with different types of VWAP-minus program: ASRs with fixed number of shares, ASRs with fixed notional, and profit-sharing contracts. The results we obtain are in line with both intuition and previous studies. The interest of our method lies in the fact that almost any price dynamics can be considered and that new types of contract can be handled. In particular, we manage to handle contracts for which classical methods usually fail because of (i) high dimensionality and (ii) the very complexity of some contracts that cannot be written as payoffs.

Appendix

In this appendix, we propose a brief introduction to neural networks and their use for approximating functions. We also briefly expose their interest for solving optimisation problems. Finally, we describe the networks used in this paper with a focus on their recurrent structure.

A bit of history

In the 1950s, decisive results were obtained regarding representations of real continuous functions of several variables through addition and composition of continuous functions depending on a smaller number of variables. First, Kolmogorov obtained in [26] a representation with functions of three variables. Then, Arnold, his 19-year-old student at that time, obtained a representation with functions of two variables (see [5]), thus providing an answer to the continuous (as opposed to algebraic) version of Hilbert’s thirteenth problem. Finally, Kolmogorov derived in [27] his celebrated superposition theorem stating the existence of a representation by superpositions of continuous functions of one variable:

Theorem 1** (Kolmogorov’s superposition theorem).**

Let $n\geq 2$ be an integer. There exist real continuous functions $(\phi_{p,q})_{1\leq p\leq n,1\leq q\leq 2n+1}$ defined on $[0,1]$ such that for any real continuous function $f$ defined on $[0,1]^{n}$, there exist real continuous functions $(\chi_{q})_{1\leq q\leq 2n+1}$ defined on $[0,1]$ such that

[TABLE]

This result has then been improved in many ways, from reducing the number of necessary functions, to imposing monotonicity, Lipschitz, or Hölder conditions on the functions. However, despite constructive proofs of superposition theorems, real computation of representations remains almost always impossible because constructions always involve limits, hence infinite loops. Furthermore, the functions involved are often unreasonably complicated because the goal is to obtain a representation rather than an approximation.

Function approximation with neural networks

In fact, an important strand of research has been dedicated to obtain approximate representations with simple functions of a single variable. In particular, feedforward neural networks are often regarded as good candidates for approximating nonlinear functions.

A feedforward neural network is a function of the form $\Psi^{L}$ (for $L\geq 1$ an integer) where $(\Psi^{l})_{0\leq l\leq L}$ are defined recursively171717Layer [math] is called the input layer while layer $L$ is called the output layer. by $\Psi^{0}:x\in\mathbb{R}^{d_{0}}\mapsto x$ and

[TABLE]

where for each layer $l\in\{1,\ldots,L\}$, $g^{l}:(x_{1},\ldots,x_{d_{l}})\in\mathbb{R}^{d_{l}}\mapsto(h^{l}(x_{1}),\ldots,h^{l}(x_{d_{l}}))\in\mathbb{R}^{d_{l}}$ applies elementwise either the identity function or a nonlinear function called activation function (in the latter case $h^{l}$ is typically a sigmoid function, the hyperbolic tangent function, a softmax function, or the rectified linear unit (ReLU) function – i.e. $x\in\mathbb{R}\mapsto x\mathbf{1}_{x\geq 0}$), $A^{l}\in\mathbb{R}^{d_{l}\times d_{l-1}}$ is a matrix of weights, and $b^{l}\in\mathbb{R}^{d_{l}}$ a vector of weights also called bias. A feedforward neural network is often denoted by $\Psi_{\theta}$ where $\theta=(A_{l},b_{l})_{1\leq l\leq L}$ stacks all the parameters.

In addition to the link with neurons and the functioning of the animal/human brain (on which we shall remain silent throughout this short appendix), the main interest of feedforward neural networks lies in universal approximation theorems such as the one proved by Hornik in [22]. In a nutshell, these theorems state, under various – and usually mild – hypotheses, that any real function of several variables (here $d_{0}$ variables) can be approximated by a feedforward neural network of the above form, provided that there are sufficiently many neurons (i.e. $d_{j}$ for $j\in\{1,\ldots,J\}$ large enough). More precisely, Hornik [22] proved the following universal approximation theorem for continuous functions:181818Hornik also proved a $L^{p}$ version of the universal approximation theorem. Many other versions of this theorem exist, for instance to handle ReLU activation functions which are unbounded.

Theorem 2** (Hornik’s universal approximation theorem).**

Let $K$ be a compact set of $\mathbb{R}^{d_{0}}$. Let $h$ be a real-valued continuous, bounded, and nonconstant function defined on $\mathbb{R}$. Then

[TABLE]

is dense in the set $C(K)$ of real continuous functions defined on $K$.

Most of early universal approximation theorems state that one hidden layer with an activation function and a second output layer is sufficient to obtain a good approximation. However, and this is why deep learning is so important, it is often more efficient to approximate a function with several layers consisting of a few neurons than with one layer with a lot of neurons.

Remark 7**.**

It is noteworthy that, in this paper, neural networks with one hidden layer and a few dozens of neurons were sufficient to obtain satisfactory results.

Optimisation with neural networks

Feedforward neural networks allow to approximate a large class of functions but so do other families of functions. The reasons why they are often favored have to do with (i) the definition (or characterisation) of the functions to be approximated in machine learning, (ii) the possibility to compute $\Psi_{\theta}(x)$ and its gradient with respect to $\theta$ in an efficient way using forward and backward propagation respectively (see for instance [16] for an introduction), and (iii) the fact that approximations with feedforward neural networks of functions defined on a high-dimensional space are often parsimonious, hence the advantage of neural networks over other families of functions in front of the curse of dimensionality.

In machine learning indeed, be it for supervised, unsupervised, or reinforcement learning, the problem often boils down to approximating a function defined as the solution to an optimisation problem. Therefore, the ability to efficiently differentiate the function with respect to its parameters is essential to use all the classical techniques of gradient descent.191919The examples of this paper have been computed using the open-source package TensorFlow, distributed by Google.

Recurrent network structure

In this paper we use neural networks to approximate two functions: the optimal trading strategy of the trader and its optimal stopping decision in the form of a probability to exercise the option. These functions depend, in general, on the time to maturity, the cash spent since inception, the current price of the stock, its running average since the beginning of the contract, and the number of shares already acquired. The problem is discretised in time and decisions made at any time step influence the state at all future time steps including the final one. In particular, the learning procedure to optimise the mean-variance objective function requires to take into account all subsequent effects when computing the derivatives with respect to the weights of the neural networks involved in the decisions taken at a time period $n<N-1$.

Formally speaking, the trading decision made at a period $n<N-1$, i.e. $v_{\theta}(n,S_{n},A_{n},X_{n},q_{n})$ where $\theta$ stands as always for the parameters of the neural network, allows to compute the next value of the inventory process, i.e. $q_{n+1}=q_{n}+v_{\theta}(n,S_{n},A_{n},X_{n},q_{n})\delta t$, which enters as an input in the network to compute the next trading decision $v_{\theta}(n+1,S_{n+1},A_{n+1},X_{n+1},q_{n+1})$, and so on (in the case of the profit-sharing contract the same problem occurs with the cash variable). As a consequence, the networks involved in our approach are recurrent. This does not however raise any technical difficulty for computing gradients because we use automatic differentiation.202020Recurrent structures are sometimes prone to vanishing gradient problems. However, we never noticed any such problem while using our methods.