Stochastic hybrid differential games and match race problems

TL;DR

This paper introduces a stochastic hybrid differential game framework applied to sailing match races, providing a convergent approximation method for the game's value function and validating it through typical racing scenarios.

Contribution

It develops a novel stochastic hybrid differential game model for sailing races and offers a convergent numerical scheme for computing the game's value function.

Findings

Validated the approximation scheme on typical racing scenarios

Demonstrated the model's ability to capture strategic interactions in sailing races

Provided a computational method for analyzing match race strategies

Abstract

We discuss the general framework of a stochastic two-player, hybrid differential game, and we apply it to the modelling of a "match race" between two sailing boats, namely a competition in which the goal of both players is to proceed in the windward direction, while trying to slow down the other player. We provide a convergent approximation scheme for the computation of the value function of the game, and we validate the approach on some typical racing scenarios.

Equations

$$\mathcal{A}=\left\{a:(0,\infty )\to A|a\text{ measurable}\right\},$$

$$\mathcal{B}=\left\{b:(0,\infty )\to B|b\text{ measurable}\right\}.$$

$$\mathcal{Q}=\left\{Q:(0,\infty )\to \mathcal{I}|Q(t)=\sum _{i\ge 0}{q}_i{\chi }_{[t_i,t_{i+1})}(t)\right\},$$

$$\mathcal{R}=\left\{R:(0,\infty )\to \mathcal{J}|R(t)=\sum _{i\ge 0}{r}_i{\chi }_{[t_i,t_{i+1})}(t)\right\},$$

$$\{\begin{array}{l}dX(t)=f(X(t),Q(t),a(t),R(t),b(t))dt+\sigma (X(t),Q(t),R(t))d{W}_t,t>0,\\ X(0)=x,Q(0^{+})=q,R(0^{+})=r,\end{array}$$

$$X(t)=x+{\int }_0^{t}f(X(s),Q(s),a(s),R(s),b(s))ds+{\int }_0^t\sigma (X(s),Q(s),\bar{R}(s))d{W}_s.$$

$$\begin{gathered} J(x,q,r;\alpha ,\beta ):=\mathbb{E}({\int }_0^{+\infty }{e}^{-\lambda s}\ell (X(s),Q(s),a(s),R(s),b(s))ds \\ +\sum _{i\ge 0}{e}^{-\lambda t_i}\left[C_A\left(Q(t_i^{-}),Q(t_i^{+})\right)+C_B\left(R(t_i^{-}),R(t_i^{+})\right)\right]). \end{gathered}$$

$$C_A(q,q)=0\text{mbox}foreveryq\in \mathcal{I},C_B(r,r)=0\text{mbox}foreveryr\in \mathcal{J}.$$

$$\underset{q_1\ne q_2}{\max }{C}_A(q_1,q_2)\le -C_0,\underset{r_1\ne r_2}{\min }{C}_B(r_1,r_2)\ge C_0.$$

$$\beta (s)=\tilde{\beta }(s)\text{mbox}foralls\le t\text{mbox}implies\varphi [\beta ](s)=\varphi [\tilde{\beta }](s)\text{mbox}foralls\le t.$$

$$\text{mbox}(resp.\alpha (s)=\tilde{\alpha }(s)\text{mbox}foralls\le t\text{mbox}implies\psi [\alpha ](s)=\psi [\tilde{\alpha }](s)\text{mbox}foralls\le t.)$$

$$\underline{v}(x,q,r):=\underset{\psi \in \Psi }{\inf }\underset{\alpha \in \mathcal{Q}\times \mathcal{A}}{\sup }J(x,q,r;\alpha ,\psi [\alpha ]),$$

$$\overline{v}(x,q,r):=\underset{\varphi \in \Phi }{\sup }\underset{\beta \in \mathcal{R}\times \mathcal{B}}{\inf }J(x,q,r;\varphi [\beta ],\beta ).$$

$$\underline{v}(x,q,r)=\inf_{\psi\in\Psi}\sup_{\alpha\in{\mathcal{Q}}\times{\mathcal{A}}}\Big{\{}{\mathbb{E}}\Big{(}\int_{0}^{\tau}\ell(X(s),\alpha(s),\psi[\alpha](s))ds\\ +\sum_{t_{i}<\tau}e^{-\lambda t_{i}}\left[C_{A}\left(Q(t_{i}^{-}),Q(t_{i}^{+})\right)+C_{B}\left(R(t_{i}^{-}),R(t_{i}^{+})\right)\right]\\ +\underline{v}(X(\tau),Q(\tau),R(\tau))e^{-\lambda\tau}\Big{)}\Big{\}},$$

$$\mathcal{N}[\phi ](x,q,r):=\underset{\hat{q}\ne q}{\max }\{\phi (x,\hat{q},r)+C_A(q,\hat{q})\},$$

$$\mathcal{M}[\phi ](x,q,r):=\underset{\hat{r}\ne r}{\min }\{\phi (x,q,\hat{r})+C_B(r,\hat{r})\},$$

$$\mathcal{N}[\underline{v}](x,q,r)\le \underline{v}(x,q,r)\le \mathcal{M}[\underline{v}](x,q,r).$$

$$H^{-}(x,q,r,p):=\underset{a\in A}{\min }\underset{b\in B}{\max }\{-f(x,q,a,r,b)\cdot p-\ell (x,q,a,r,b)\},$$

$$H^{+}(x,q,r,p):=\underset{b\in B}{\max }\underset{a\in A}{\min }\{-f(x,q,a,r,b)\cdot p-\ell (x,q,a,r,b)\},$$

$$F^{-}[\phi ](x,q,r)=\lambda \phi (x,q,r)+H^{-}(x,q,r,D\phi )-\frac{1}{2}\mathrm{tr}\left(\sigma {\sigma }^T{D}^2\phi (x,q,r)\right),$$

$$F^{+}[\phi ](x,q,r)=\lambda \phi (x,q,r)+H^{+}(x,q,r,D\phi )-\frac{1}{2}\mathrm{tr}\left(\sigma {\sigma }^T{D}^2\phi (x,q,r)\right),$$

$$\max \left\{\underline{v}-\mathcal{M}[\underline{v}],\min \left\{\underline{v}-\mathcal{N}[\underline{v}],F^{-}[\underline{v}]\right\}\right\}=0,$$

$$\max \left\{\overline{v}-\mathcal{M}[\overline{v}],\min \left\{\overline{v}-\mathcal{N}[\overline{v}],F^{+}[\overline{v}]\right\}\right\}=0,$$

$$\sum _{i=1}^N\left\{C_A(q_i,q_{i+1})+C_B(r_i,r_{i+1})\right\}\ne 0.$$

$$d{X}_3(t)=d\Theta (t)=\sigma dW(t),$$

$$\{\begin{array}{l}{\dot{X}}_1(t)=s(a^{\ast })\sin \left(\Theta (t)+(-1{)}^{Q(t)}{a}^{\ast }\right)\\ {\dot{X}}_2(t)=s(a^{\ast })\cos \left(\Theta (t)+(-1{)}^{Q(t)}{a}^{\ast }\right).\end{array}$$

$$\left\{\begin{array}[]{l}dX^{A}(t)=f^{A}(X(t),\Theta(t),Q(t))dt\\ dX^{B}(t)=f^{B}(X(t),\Theta(t),R(t))dt\\ d\Theta(t)=\sigma dW(t)\end{array}\right.\qquad\left\{\begin{array}[]{l}X^{A}(0)=x^{A}\\ X^{B}(0)=x^{B}\\ \Theta(0)=\theta\end{array}\right.$$

$$f^A(x,\theta ,q)=s^A(x,\theta )\left(\sin (\theta +(-1{)}^q{a}^{\ast }),\cos (\theta +(-1{)}^q{a}^{\ast })\right),$$

$$f^B(x,\theta ,r)=s^B(-x,\theta )\left(\sin (\theta +(-1{)}^r{b}^{\ast }),\cos (\theta +(-1{)}^r{b}^{\ast })\right),$$

$$s^P(x,\theta )={\bar{s}}^P\left(1+\min \{s_0^P(x\cdot (\sin (\theta ),\cos (\theta ))e^{-s_1^P|x{|}^2},0\}\right)(P=A,B),$$

$${\bar{s}}^A(x,\theta ,q)={\bar{s}}^A{\left(1-{\nu }_1{e}^{-\frac{x^2}{{\nu }_2}}\right)}^{q-1}$$

$$C_{A}(q_{1},q_{2})=\left\{\begin{array}[]{ll}-C^{A}&\mbox{if }q_{1}\neq q_{2}\\ 0&\mbox{otherwise,}\end{array}\right.\qquad C_{B}(r_{1},r_{2})=\left\{\begin{array}[]{ll}C^{B}&\mbox{if }r_{1}\neq r_{2}\\ 0&\mbox{otherwise.}\end{array}\right.$$

Full text

Stochastic hybrid systems, differential games, Hamilton–Jacobi equations

93E20, 49N70, 34K34, 65N06

1 Introduction

Hybrid processes are present in many economic and technological systems, whose dynamics can be modelled by a collection of controlled ordinary or stochastic differential equations: besides the standard actions performed on the current dynamics at a given time, the controller also has the option to switch to a different dynamics, in order to optimize some objective functional.

Starting from the late 90s, several attempts have been made to provide a precise notion of hybrid systems. Among the different concepts proposed, we quote here [1] and [2] for respectively the deterministic and the stochastic case. The common feature of these models is to consider an extended state space for the dynamics, given by the product of both a continuous component and a discrete component, the latter indexing the active dynamics within a finite set.

In the optimal control of such class of systems, dynamic programming techniques have been widely investigated in the literature. The formulation of the problem in terms of a Bellman equation leads to a system of quasi-variational inequalities, which involve two different Bellman operators, related to respectively the continuous and the discrete control actions. A theoretical study of the problem in the framework of viscosity solutions can be found in [2, 3]. The numerical treatment via monotone schemes has also been studied in [4], proving that the classical Barles–Souganidis theory [5] applies to the hybrid case, and providing a convergent technique to construct asymptotically optimal controls.

On the other hand, the case of differential games in the presence of hybrid dynamics seems much less explored in the literature. To our knowledge, the first study of a deterministic game under pure switching controls is given in [6]. Using the celebrated notion of non-anticipating strategies by Elliott and Kalton [7], one can prove the existence of a value for the game under a technical assumption, the so-called no free loop property. The unique value function satisfies a dynamic programming principle and can be characterized as the viscosity solution of an Isaacs system of quasi-variational inequalities. We refer to this work also for an extensive review of the earlier literature on the subject. A more recent and general study, still in the deterministic case but much in the spirit of hybrid systems, is provided in [8]. Requiring also the classical Isaacs conditions, the existence of a value is proved for games involving continuous, impulsive and discrete controls. Finally, concerning the stochastic case, we refer to some recent papers [9, 10, 11, 12, 13, 14].

In this paper, we use the theory of stochastic hybrid differential games to model a route planning problem for two competing sailing boats, known as a match race. In this problem, the aim of the two competitors is to reach a mark at the end of a race leg before the other, regardless of the time to reach the goal. This aspect makes the problem quite different from optimal navigation (discussed, e.g., in [15, 16, 17]) and motivates the use of game theory to model the interaction mechanics between the boats. This issue was also addressed in [18], where the authors propose a technique to assess virtual competitions between yachts and to evaluate the pros and cons of various race scenarios. The physical interaction between the two boats (the wind shadow region where one boat perturbs the wind) is modelled using a penalization/reward term. Other works related are [19, 20], where a risk model is included in the strategic decision process, and it is shown that, rather than finding the strategy that minimizes the time to complete the race leg, a strategy aimed at maximizing the probability of completing before the opponent offers better chances of victory. In the same works, the authors use a short term wind forecast methodology based on Artificial Neural Networks to model the instability of the wind. The originality of the methodology that we propose in this paper lies on a game-theory-based formulation of the interaction between the two boats. This framework permits the observation of highly sophisticated strategic choices that are commonly used by tacticians in match race competitions, and a precise timing and quantification of them.

The paper is organized as follows. In Section 2, we introduce the mathematical framework for a stochastic hybrid game, reporting some results concerning its well-posedness in the viscosity sense, as well as the conditions for the existence of a value. In Section 3, we discuss in detail our game, and we analyze some relevant features of the corresponding value function. Section 4 is devoted to the numerical solution of the Isaacs system of the game. We build a convergent algorithm based on a suitable monotone scheme, and we provide some hints on its actual implementation. Finally, in Section 5, we perform some numerical tests, showing the effectiveness of the technique in different scenarios of application.

2 Stochastic hybrid differential games

We describe the general structure of a zero-sum stochastic hybrid differential game, and we report the main results concerning the well-posedness of the problem in the sense of viscosity solutions. The following presentation is a modified version of the one proposed in [6]. Fundamental contributions are also [2, 1] adapted to the stochastic case as in [8] or in the same spirit of [21]. We refer to these papers for further details and rigorous proofs.

Given two compact sets $A\subset{\mathbb{R}}^{m_{A}}$ and $B\subset{\mathbb{R}}^{m_{B}}$ (for some integers $m_{A}$, $m_{B}$), we define the following standard sets of continuous controls for the two players, respectively

[TABLE]

[TABLE]

Moreover, in order to model the possibility for the two players to switch between different dynamics, we consider two finite sets of indices ${\mathcal{I}}=\{1,2,\ldots,N_{\mathcal{I}}\}$ and ${\mathcal{J}}=\{1,2,\ldots,N_{\mathcal{J}}\}$ (for some integers $N_{\mathcal{I}}$, $N_{\mathcal{J}}$), and we define the following sets of piecewise constant discrete controls, respectively

[TABLE]

[TABLE]

where $\{t_{i}\}$ is the sequence of (ordered) switching times, $\{q_{i}\}\subset{\mathcal{I}}$, $\{r_{i}\}\subset{\mathcal{J}}$ are the corresponding sequences of switching values for the two players, and $\chi_{[t_{i},t_{i+1})}$ denotes the characteristic function of the interval $[t_{i},t_{i+1})$.

We consider the dynamical system described by the following controlled stochastic differential equation (SDE):

[TABLE]

where (for some integers $d,k$) $x,X\in{\mathbb{R}}^{d}$, $q\in{\mathcal{I}}$, $r\in{\mathcal{J}}$, $a(\cdot)\in\mathcal{A}$, $Q(\cdot)\in{\mathcal{Q}}$, $b(\cdot)\in\mathcal{B}$, $R(\cdot)\in{\mathcal{R}}$, while $f:{\mathbb{R}}^{d}\times{\mathcal{I}}\times A\times{\mathcal{J}}\times B\to{\mathbb{R}}^{d}$ is the dynamics, $dW_{t}$ is the differential of a $k$-dimensional Brownian process, and $\sigma:{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}\to{\mathbb{R}}^{d\times k}$ is the corresponding covariance matrix.

To properly define solutions of the stochastic differential equation (1), we need a standard regularity assumption:

• H1 -

$f$ and $\sigma$ are globally bounded and uniformly Lipschitz continuous with respect to $x$.

Then, the following integral representation formula holds:

[TABLE]

The stochastic trajectory starts from $(x,q,r)$ in the extended state space ${\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}$. At each time $t>0$ the first player can act on the current dynamics through the control $a(\cdot)\in\mathcal{A}$, or switch to another dynamics using the discrete control $Q(\cdot)\in{\mathcal{Q}}$. Similarly, the second player employs the controls $b(\cdot)\in\mathcal{B}$ and $R(\cdot)\in{\mathcal{R}}$. This setting is suitable for our application to a match race competition, but we remark that the most general framework of hybrid control systems (see, e.g., [2]), allows one to deal with problems including also autonomous transitions and jumps in the state $X$.

Now, we define the game between the two players. To this end, let us introduce a more compact notation for the controllers, by setting respectively $\alpha(t):=(Q(t),a(t))\in{\mathcal{Q}}\times{\mathcal{A}}$ and $\beta(t):=(R(t),b(t))\in{\mathcal{R}}\times{\mathcal{B}}$. Moreover, we consider the following cost functional:

[TABLE]

Here, the symbol ${\mathbb{E}}$ denotes expectation with respect to the Wiener measure, while the first integral term defines a standard infinite horizon functional, with discount factor $\lambda>0$ and a running cost $\ell:{\mathbb{R}}^{d}\times{\mathcal{Q}}\times{\mathcal{A}}\times{\mathcal{R}}\times{\mathcal{B}}\to{\mathbb{R}}$. We assume that:

• H2 -

$\ell$ is non-negative, bounded and uniformly Lipschitz continuous with respect to $x$.

On the other hand, the second term in (2) accounts for the discounted costs $C_{A}:{\mathcal{I}}\times{\mathcal{I}}\to{\mathbb{R}}$ and $C_{B}:{\mathcal{J}}\times{\mathcal{J}}\to{\mathbb{R}}$ associated to the switches of the two players ($A$ and $B$ respectively) at times $\{t_{i}\}$. Here, player $A$ wants to maximize $J$ using the control $\alpha$, thus paying a negative cost $C_{A}$ for each switch. Similarly, player $B$ wants to minimize $J$ using the control $\beta$ and paying a positive cost $C_{B}$ for each switch. Note that, to simplify notation, we regrouped the switching times of both players in a single sequence $\{t_{i}\}$. This means that, if only one player performs a switch at time $t_{i}$, the corresponding cost of the other player should be zero. We summarize all these properties by requiring the following assumptions:

• H3 -

$C_{A}$ and $C_{B}$ are bounded and satisfy

[TABLE]

Moreover, there exists $C_{0}>0$ such that

[TABLE]

We proceed by defining the value functions of the game. To this end, we employ the classical notion of non-anticipating strategies [7, 6], which allows to rigorously prove a dynamic programming principle.

Definition 1

A non-anticipating strategy for player $A$ (resp. $B$) is a map $\phi:{\mathcal{R}}\times\mathcal{B}\rightarrow{\mathcal{Q}}\times\mathcal{A}$ (resp. $\psi:{\mathcal{Q}}\times\mathcal{A}\rightarrow{\mathcal{R}}\times\mathcal{B}$) such that, for any $t>0$,

[TABLE]

[TABLE]

We denote the set of non-anticipating strategies by $\Phi$ for player $A$, and by $\Psi$ for player $B$. Then, for every $(x,q,r)\in{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}$, we define the lower value function $\underline{v}$ of the game as

[TABLE]

and the upper value $\overline{v}$ as

[TABLE]

Moreover, if $\underline{v}\equiv\overline{v}$, we say that the game has a value, and we denote it by $v$.

In the next Proposition, we state the dynamic programming principle satisfied by both the value functions.

Proposition 1

Under the assumptions H1-H3, for all $(x,q,r)\in{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}$ and $\tau>0$, the following equation holds true

[TABLE]

where $Q$ and $R$ are the switching controls contained respectively in the strategy $\alpha$ and $\psi[\alpha]$. A similar equation holds for the upper value function $\overline{v}$, by swapping the role between $\inf$ and $\sup$ in (5).

Now, for a generic function $\varphi:{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}\to{\mathbb{R}}$, we define the two following switching operators

[TABLE]

which provide some natural bounds on the value functions, as stated in the next Proposition.

Proposition 2

For every $(x,q,r)\in{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}$ the lower value function $\underline{v}$ satisfies

[TABLE]

The same estimates hold for the upper value function $\overline{v}$.

Proposition 1 and Proposition 2 allow to derive the Hamilton–Jacobi–Isaacs equations of the game. More precisely, for $x,p\in{\mathbb{R}}^{d}$, $q\in{\mathcal{I}}$ and $r\in{\mathcal{J}}$, we introduce the Hamiltonians

[TABLE]

and the second order differential operators

[TABLE]

where $D$ and $D^{2}$ denote respectively the gradient and the hessian with respect to $x$, $\sigma^{T}$ is the transpose of $\sigma$, and $\operatorname{\text{tr}}(\cdot)$ stands for the matrix trace.

Then, it follows that the value functions $\underline{v}$ and $\overline{v}$ satisfy, for every $(x,q,r)\in{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}$, respectively

[TABLE]

and

[TABLE]

namely two systems of $N_{\mathcal{I}}N_{\mathcal{J}}$ quasi-variational inequalities. In each system, we can identify three separate operators, which provide respectively the best possible switching for the two players, and the best possible continuous controls. The arguments attaining the respective extrema in such equations represent the overall optimal control strategies. The derivation of (10) and (11), which is elementary under differentiability assumptions, can be rigorously justified in a more general setting by an adaptation of the viscosity theory [3] to the case under consideration.

To conclude this section, we briefly discuss the key steps for proving the existence of a value for the game, namely that $\underline{v}\equiv\overline{v}$. First, a uniqueness result for the viscosity solutions of both (10) and (11) is needed. In this direction, the following additional assumption, the so-called no free loop property, appears in several papers on hybrid games, see e.g., [14, 6, 10]:

• H4 -

Let $\{(q_{i},r_{i})\}_{i=1,...,N+1}$ be a finite sequence of indices such that $(q_{i},r_{i})\not=(q_{i+1},r_{i+1})$ for every $i=1,...,N-1$ and $(q_{1},r_{1})=(q_{N+1},r_{N+1})$. Then

[TABLE]

Although technical, this assumption seems unavoidable in order to obtain a comparison principle between a viscosity sub-solution $u$ and a viscosity super-solution $w$ of (10) (the same reasoning applies to (11)). The idea is that, using assumption H4, one can find, for every $x\in{\mathbb{R}}^{d}$, a common state $(q^{*},r^{*})\in{\mathcal{I}}\times{\mathcal{J}}$ in which both inequalities $F^{-}[u](x,q^{*},r^{*})\leq 0$ and $F^{-}[w](x,q^{*},r^{*})\geq 0$ hold. Then, one can proceed with the usual comparison of the Hamiltonians in the viscosity theory and conclude that $u\leq w$. This result implies that $\underline{v}$ is the unique viscosity solution of (10) and $\overline{v}$ is the unique viscosity solution of (11).

Finally, the existence of a value for the game is guaranteed by providing assumptions that let the Isaacs systems (10) and (11) coincide, as for the following classical Isaacs conditions:

• H5 -

$H^{-}(x,q,r,p)=H^{+}(x,q,r,p)$ for every $(x,q,r,p)\in{\mathbb{R}}^{d}\times{\mathcal{I}}\times{\mathcal{J}}\times{\mathbb{R}}^{d}$.

Summarizing, we have the following result.

Theorem 1

Under assumption H1-H5, the value function $v:=\underline{v}\equiv\overline{v}$ is the unique viscosity solution of both (10) and (11).

3 The match race problem

We apply the theoretical framework of hybrid differential games, discussed in the previous section, to a real-world application. A match race is a competition between two sailing boats, in which the goal of both players is to reach, as first, the end of the course, regardless of their relative distance. Each player can take advantage of the wind fluctuations to proceed upwind towards the finish line, by adjusting the relative angle between the sail and the wind, and also changing the tack side. In addition, the players can make use of their respective influence, caused by the sail turbulence. This is usually an advantage for the leading boat, which can exploit this influence to control the other player. Note that a reasonable description of this problem requires, at least, a state space of dimension $d=5$: two pairs of coordinates to track the positions of the boats in a plane, and one coordinate for the wind angle. Here, we consider a simplified game, namely we neglect the windward mark, and we just focus on the strategies of the two players in the space of relative positions. This reduces the problem to dimension $d=3$, but it is still a realistic racing criterion when the two players are far from the windward mark. Moreover, it can be numerically solved in a reasonable time also on a laptop computer. The analysis and the parallel implementation of the full game is under investigation, and will be addressed in a forthcoming paper.

In the next subsections, we first introduce the hybrid dynamics for the boats, following the model presented in [17]. Then, we define the hybrid game, by suitably setting all the parameters appearing in the cost functional (2). Finally, we present a more detailed analysis of the value function of the game, in case the two players are far enough from each other.

3.1 Dynamics modelling

We consider the motion on a plane of a single boat, subject to a wind of constant speed and variable direction. We set the dimension of the state space to $d=3$, in which the first two components $x_{1}$ and $x_{2}$ represent the position of the boat, while the third component $x_{3}$ gives the angle $\theta\in[-\pi,\pi]$ of the wind with respect to the vertical axis. In particular, $\theta$ is negative in the second and third quadrant, and positive in the first and fourth quadrant, see Figure 1a.

Moreover, we assume that the wind has a purely Brownian nature, i.e., it evolves according to the one-dimensional SDE:

[TABLE]

where $dW$ denotes the differential of a Brownian process, and $\sigma>0$ is the corresponding standard deviation.

On the other hand, the motion of the boat results from both the wind direction and the boat characteristics. Following the notation presented in Section 2, we introduce the set of admissible controls $A=[0,\pi]$ as the unsigned angles between the boat direction and the wind, so that the continuous control is given by a function $a:[0,+\infty)\to A$. Then, since the wind speed is constant, the boat speed will depend only on the angle $a$, by means of a function $s:A\rightarrow{\mathbb{R}}_{+}$, the so-called polar plot of the boat. Figure 1a summarizes this geometric setting, while in Figure 1b we show a typical form of the polar plot, with the whole set of speeds associated with the port tack. Note that, for $a=0$, the trajectory points directly in the upwind direction, whereas, for $a=\pi$, the trajectory has the same direction of the wind field.

When sailing to windward, it is customary to keep constantly the boat at its most efficient angle with the wind, that is, at the angle $a^{*}\approx\frac{\pi}{4}$, corresponding the largest windward component of the speed. In this case, the dynamics can be simplified by freezing the control at the value $a^{*}$ (see Figure 1c), and acting on the system only by changing tack. In what follows, we will use this simplified approach.

Finally, we introduce the discrete control, namely a piecewise constant function $Q:[0,+\infty)\to{\mathcal{I}}$, taking values in the discrete set ${\mathcal{I}}=\{1,2\}$. The two possible discrete states correspond to the tack sides, where the port tack is identified by $q=1$ and the starboard tack by $q=2$. Hence, the dynamics of the boat is given by

[TABLE]

3.2 Game modelling

We define the game in reduced coordinates, i.e., we consider as space variable the relative position of the two players. We denote by $x^{A}=(x_{1}^{A},x_{2}^{A})\in{\mathbb{R}}^{2}$, $x^{B}=(x_{1}^{B},x_{2}^{B})\in{\mathbb{R}}^{2}$ and $\theta\in[-\pi,\pi]$, respectively the coordinates of the two players and the wind angle, while the reduced coordinates are given by $x=x^{A}-x^{B}\in{\mathbb{R}}^{2}$. Then, for $q,r\in\mathcal{I}=\mathcal{J}=\{1,2\}$ and discrete controls $Q,R:[0,+\infty]\to{\mathcal{I}}$ such that $Q(0)=q$ and $R(0)=r$, we define the controlled dynamics of the game according to (12) and (13):

[TABLE]

where

[TABLE]

[TABLE]

with $a^{*}=b^{*}=\frac{\pi}{4}$. The speed functions $s^{A}$ and $s^{B}$ contain the information about the interaction between the two players. As an example, we can take

[TABLE]

for given positive constants $\bar{s}^{P},s_{0}^{P},s_{1}^{P}$, which would model the wind shadow region, i.e., a situation in which the player $P$ has its maximum speed $\bar{s}^{P}$ when the two players are far from one another, but it is slowed down when its position is close and behind or on the downwind side of the other (note the dependency of $f^{A}$ on $x$ and of $f^{B}$ on $-x$, which reflects the speed profile with respect to the origin, according to the leading player). Figure 2 shows the level sets of the speed function $s^{P}$ in (15), corresponding to $\bar{s}^{P}=0.05$, $s_{0}^{P}=20$ and $s_{1}^{P}=300$, for $\theta=\frac{\pi}{4}$.

We remark that we can also incorporate in the maximum speeds $\bar{s}^{A}$ and $\bar{s}^{B}$ an additional dependency on the states $(x,q,r)$. This can be useful to model the rules about the right of way in match race competitions. For instance, with a little abuse of notation, we can choose

[TABLE]

(and similarly $\bar{s}^{B}(x,\theta,r)$) to introduce a penalization of the speed of the boat on the port-tack ($q,r=2$), which activates only when the distance between the boats is small enough, tuned by the choice of the parameters $\nu_{1},\nu_{2}$. This penalization mimics the interaction of a boat meeting a competitor on opposite tacks (see [22, Rule 10]).

In order to define the infinite horizon cost functional $J$ in (2), we choose two constants $C^{A},C^{B}>0$, and we set the switching costs as follows

[TABLE]

Finally, we choose the running cost

[TABLE]

so that the cost functional integrates the vertical component of the relative speed of the two boats. This results in a game in which each player wants to overcome the opponent along the vertical component with the least number of switches. Due to the lack of continuous controls, the corresponding systems of quasi-variational inequalities (10)-(11) coincide, and take the form:

[TABLE]

where, for every $q,r\in{\mathcal{I}}={\mathcal{J}}=\{1,2\}$, we set $\hat{q}=3-q$ and $\hat{r}=3-r$, and we denoted by $f$ the deterministic part of the coupled dynamics in ${\mathbb{R}}^{d}$, namely

[TABLE]

We can observe that, in the present setting, the technical assumption H4 in Theorem 1 is satisfied if $C_{A}\neq C_{B}$. Otherwise, uniqueness of a solution is not ensured.

3.3 Decoupling of the game with “far” players

As discussed in the previous section, the coupling in the dynamics of the two players, and hence the essence of the game, is entirely embedded in the speed function $s^{P}$. A key observation is that if the two players are far enough from each other, i.e., if $|x|\gg 1$, then $s^{P}\approx\bar{s}^{P}$. Consequently, a fair approximation of the far dynamics of each player depends only on the wind direction and on the switching strategy. In this setting, we can provide a more explicit analysis of the game, and also obtain suitable boundary conditions for the approximation of the problem in a bounded domain, as it will be discussed later. We remark that this analysis is much in the same spirit of the one carried out in [23, Chapter 5], and in some sense brings it to its final conclusions, in the case in which the player is far from the target.

Assuming that $|x|=|x^{A}-x^{B}|\gg 1$, and using the definition of $s^{P}$ and $\ell$, we can split the cost functional $J$ in (2) as the difference

[TABLE]

where

[TABLE]

[TABLE]

As a consequence, we get

[TABLE]

where, for $P=A,B$, and $p=q,r\in{\mathcal{I}}={\mathcal{J}}=\{1,2\}$, we denote by $v^{P}(p,\theta)$ the value function corresponding to the optimal control problem, for the single player $P$, of maximizing $J^{P}$ subject to the dynamics $f^{P}$. We remark that, due to the special structure of $f^{P}$ and of the running cost in $J^{P}$, the value function $v^{P}$ depends only on $\theta$ and on the discrete state $p$. Moreover, it satisfies the following system of quasi-variational inequalities: for $p\in{\mathcal{I}}=\{1,2\}$ and $\hat{p}=3-p$,

[TABLE]

For general switching costs $C^{A}$, $C^{B}$ and speeds $\bar{s}^{A}$, $\bar{s}^{B}$, we can solve (17) numerically, as shown in the next section. Neverthless, in the symmetric case ($C^{A}=C^{B}=:\bar{c}$ and $\bar{s}^{A}=\bar{s}^{B}=:\bar{s}$), we have $v^{A}(p,\theta)=v^{B}(p,\theta)=:\bar{v}(p,\theta)$, and we can extract further information by straightforward computations. Indeed, choosing alternately $p=1$ and $p=2$ in (17), for every $\theta\in[-\pi,\pi]$ we get

[TABLE]

and, at the points $\theta$ such that both inequalities are strict, we also have the equations

[TABLE]

[TABLE]

Defining the difference $\tilde{v}(\theta)=\bar{v}(2,\theta)-\bar{v}(1,\theta)$, by linearity we readily obtain

[TABLE]

where we used the subtraction formula for the cosine function. This is a classical double obstacle problem, whose solution can be characterized as follows. First of all, the solution to the second order differential equation is given by

[TABLE]

where $C_{1},C_{2}$ are constants to be determined. By symmetry we require $v^{*}(0)=0$, whereas, imposing $C^{1}$ regularity for the contact point $\theta^{*}$ with the obstacle (this is a classical result, see [24]), we get $v^{*}(\theta^{*})=\bar{c}$ and $\frac{\partial v^{*}}{\partial\theta}(\theta^{*})=0$. This easily implies the following nonlinear equation in $\theta$:

[TABLE]

which admits a unique solution $\theta^{*}\in[0,\frac{\pi}{2}]$, since the left hand side is strictly increasing for $\theta\in[0,\frac{\pi}{2}]$ (we recall that, when sailing to windward, this is the interesting case).

Hence, we obtain

[TABLE]

and coming back to the relationship

[TABLE]

we conclude that, for $|x|\gg 1$

[TABLE]

[TABLE]

As a final remark, we point out again that, if the game is not symmetric ($C^{A}\not=C^{B}$ or $\bar{s}^{A}\not=\bar{s}^{B}$), then no such explicit computation is possible, and the single-player solution, as well as the boundary conditions for the two-player game, should be computed numerically.

Example

We solve the one dimensional problem (17) for a single player, choosing the algorithm and parameters as described in the next section. Figure 3a shows the value functions $v(1,\theta)$ and $v(2,\theta)$, corresponding to the two discrete states. We observe two crossing points, one at the origin, and one at the boundary of the periodic domain $[-\pi,\pi]$. In Figure 3b we report, for $\theta\in[-0.2,0.2]$, a detail of the difference $v(2,\theta)-v(1,\theta)$, namely, the solution of the double obstacle problem (18). The computed contact point is about $\theta^{*}=0.085722$. Finally, in Figure 3c, we show the optimal switching maps, observing a typical hysteresis loop around the origin, with optimal switching points $-\theta^{*}$ and $\theta^{*}$.

4 Numerical approximation

In this section we introduce a numerical scheme for solving the system of quasi-variational inequalities (16). To this end, it is useful to rewrite (16) in the following fixed point form:

[TABLE]

Now, given $b_{1},b_{2},b_{3}>0$, we consider the computational box $[-b_{1},b_{1}]\times[-b_{2},b_{2}]\times[-b_{3},b_{3}]$ in the reduced state space ${\mathbb{R}}^{2}\times[-\pi,\pi]$, and we introduce a uniform grid with nodes

[TABLE]

where $N$ is an integer and the space steps are given respectively by $\Delta x_{1}=2b_{1}/N$, $\Delta x_{2}=2b_{2}/N$ and $\Delta x_{3}=\Delta\theta=2b_{3}/N$.

For a generic scalar or vector function $\chi(x_{1},x_{2},\theta,q,r)$, we denote by $\chi^{i,j,k}_{q,r}$ the corresponding approximation at the point $(x_{1}^{i},x_{2}^{j},\theta^{k})$. Then, we discretize the stationary advection-diffusion equation appearing in (21) using centred differences for the second derivative of $v$, and upwind differences (according to the sign of the components of $f$) for first derivatives. By straightforward algebraic manipulations, we obtain the following scheme:

[TABLE]

where

[TABLE]

with

[TABLE]

[TABLE]

and (the symbol $\operatorname{sgn}(\cdot)$ denotes the sign of its argument)

[TABLE]

Now, we can compute the solution of (21) using fixed point iterations, as described in Algorithm 1.

Note that, in this form, the scheme is consistent, monotone and $L^{\infty}$ stable (see the analysis in [4]), and therefore convergent via the Barles–Souganidis theorem [5], in all cases in which a comparison principle holds.

We remark that the fixed point iterations are performed at the internal nodes of the grid, hence the choice of the boundary conditions for the initial guess is crucial. As discussed in the previous section, if the bounds $b_{1}$ and $b_{2}$ are large enough, the game at the boundary decouples in two optimal control problems, one for each player, both described by the same system of quasi-variational inequalities (17), in the only state variable $\theta\in[-\pi,\pi]$. These one-dimensional problems can be solved again via fixed point iterations, using the following discretization of (17) for $P=A,B$ and $p=q,r\in\{1,2\}$:

[TABLE]

with

[TABLE]

and imposing periodic boundary conditions at $\theta=\pm\pi$. Once the solutions $v^{A}$ and $v^{B}$ are computed, we set the boundary values $v^{i,j,k}_{q,r}=(v^{A})^{k}_{q}-(v^{B})^{k}_{r}$ for $i=0$ or $i=N$ or $j=0$ or $j=N$ and $0\leq k\leq N$. Note that this relation can be used also in the internal nodes, to define a reasonable initial guess and save some iterations for the convergence of Algorithm 1.

We finally remark that, in the special case $C^{A}=C^{B}$ and $\bar{s}^{A}=\bar{s}^{B}$, we can alternately solve the nonlinear equation (19) by a standard root-finding algorithm, and build the initial guess using the explicit expression (20) for the difference $v^{A}-v^{B}$.

We proceed by discussing how to build optimal trajectories for the game. With the value function $v$ at hand, we have, by construction, the following inequalities for all $i,j,k=0,...,N$, all $q,r=1,2$ and $\hat{q}=3-q$, $\hat{r}=3-r$

[TABLE]

Whenever an inequality is strict, the corresponding player keeps its discrete state, otherwise it can take an advantage on its opponent by switching to the other state and paying the corresponding cost. Then, we can easily define, for each player, an optimal switching map, depending on both the node $(x_{1}^{i},x_{2}^{j},\theta^{k})$ and the state $(q,r)$:

[TABLE]

Finally, we discretize the dynamics (14) by means of a simple forward Euler scheme with time step $\Delta t$:

[TABLE]

where $\{W_{n}\}$ is a sequence of random numbers with a normal distribution of unit variance, and

[TABLE]

define, by means of the upper integer part $\lceil\cdot\rceil$, a closest-neighbour projection on the grid of the updated state variables.

5 Numerical examples

Parameters for the simulations have been set according to the literature related to single-hull America’s Cup vessels. In what follows, the length unit amounts to 1000 meters, and the time unit to 10 seconds. We choose the bounds $b_{1}=1$, $b_{2}=1$ and $b_{3}=\frac{\pi}{4}$, with $201$ nodes for each dimension of the grid (i.e., a total number of about $3.2\cdot 10^{7}$ nodes). Concerning the boat speeds, we choose $\bar{s}^{A}=\bar{s}^{B}=0.05$ and $\bar{s}^{A}_{1}=\bar{s}^{B}_{1}=300$. For the switching costs, we consider two different settings, a symmetric case with $C^{A}=C^{B}=0.02$, and an asymmetric case with $C^{A}=0.02$ and $C^{B}=0.04$. For the wind evolution, we consider a brownian motion with standard deviation $\sigma=0.03$. Finally, we set $\lambda=0.1$ for the discount factor in the cost functional, $tol=10^{-5}$ for the convergence tolerance in Algorithm 1, and $\Delta t=0.2$ for the time step in the reconstruction of the optimal trajectories.

As already remarked, uniqueness of solutions for the system (21) is not ensured in the symmetric case $C^{A}=C^{B}$. Nevertheless, in the following tests, we always observe the convergence of the algorithm to a meaningful solution.

In the examples, we show some sample simulations obtained in typical scenarios. For each scenario, the value function and switching map have been computed in a first phase, while sample optimal (or suboptimal, as in the second example) trajectories are computed in the second phase, according to the procedure outlined in the previous section. For each simulation, four plots show respectively the wind evolution $\Theta(t)$, the resulting trajectories of the players in the $x_{1}-x_{2}$ plane, the relative position $x_{2}^{A}-x_{2}^{B}$ and the speeds of the two players, as functions of time. Trajectories and speeds are shown in red for player A, in black for player B.

Test 1

We consider the symmetric case $C^{A}=C^{B}$, and the same initial $x_{2}$-coordinate, with the player A on the left side. Figures 4–5 show two sample trajectories.

The game is led, at least up to the final time $T=1000$ of the simulation, one time for each player. Both players tend to follow the optimal single-player strategy. However, the speed plots show that, once one of the players has gained a small advantage in the first part of the game, it tries to preserve the advantage by disturbing the other player as much as possible when in favourable position, and keeping away from the other if in unfavourable position. This results in two trajectories relatively close to one another, see also the detail of the trajectories in Fig. 4.

Test 2

We still consider the symmetric case $C^{A}=C^{B}$, but here player A plays using the optimal strategy for the game, while player B plays using the single-player optimal strategy. Despite the small advantage gained by B in the first phase, A plays to disturb B (as it is apparent from the speed plot), and ends by leading the game, see Fig. 6.

Test 3

We finally consider the asymmetric case $C^{A}\neq C^{B}$. Player B starts in a favourable position, but player A ends by leading the game. Here, rather than from the coupling between the players, A seems to take advantage of its better ability to exploit wind variations, see Fig. 7.