A McKean-Vlasov approach to distributed electricity generation development

TL;DR

This paper models the interaction between centralized and distributed electricity generation using McKean-Vlasov control problems, providing explicit solutions and analyzing the impact of taxes and costs on long-term capacity and market equilibrium.

Contribution

It introduces a novel McKean-Vlasov framework for analyzing investment decisions in distributed and centralized energy, offering explicit solutions and equilibrium analysis.

Findings

Carbon tax and transmission costs justify distributed capacity growth.

Stackelberg equilibrium results in over-investment in distributed energy.

Long-term market outcomes depend on policy and cost structures.

Abstract

This paper analyses the interaction between centralised carbon emissive technologies and distributed intermittent non-emissive technologies. In our model, there is a representative consumer who can satisfy her electricity demand by investing in distributed generation (solar panels) and by buying power from a centralised firm at a price the firm sets. Distributed generation is intermittent and induces an externality cost to the consumer. The firm provides non-random electricity generation subject to a carbon tax and to transmission costs. The objective of the consumer is to satisfy her demand while mini\-mising investment costs, payments to the firm and intermittency costs. The objective of the firm is to satisfy the consumer's residual demand while minimising investment costs, demand deviation costs, and maximising the payments from the consumer. We formulate the investment decisions as McKean-Vlasov control problems with stochastic coefficients. We provide explicit, price model-free solutions to the optimal decision problems faced by each player, the solution of the Pareto optimum, and the Stackelberg equilibrium where the firm is the leader. We find that, from the social planner's point of view, the carbon tax or transmission costs are necessary to justify a positive share of distributed capacity in the long-term, whatever the respective investment costs of both technologies are. The Stackelberg equilibrium is far from the Pareto equilibrium and leads to an over-investment in distributed energy and to a much higher price for centralised energy.

Equations

$$d{X}_t^{\alpha }$$

$$J_1(\alpha )$$

$$C_1(a)$$

$$d{Q}_t^{\nu }$$

$$J_2(\nu ;X^{\alpha })$$

$$C_2(u)$$

$${\pi }_t^{\alpha ,\nu }$$

$$J(\alpha ,\nu )$$

$$V^C$$

$$f_t(x,\bar{x},a)$$

$$\rho$$

$$\displaystyle\mathbb{E}\Big{[}\int_{0}^{\infty}e^{-\rho t}|X_{t}^{\alpha}|^{2}dt\Big{]}$$

$$\displaystyle\mathbb{E}\Big{[}\int_{0}^{\infty}e^{-\rho t}|P_{t}|^{2}dt\Big{]}$$

$$\displaystyle\mathbb{E}[v_{0}(x_{0},x_{0})]\;=\;\mathbb{E}[V_{0}^{\alpha}]\;=\;\mathbb{E}[S_{0}^{\alpha}]\leq\mathbb{E}[S_{T}^{\alpha}]\;=\;\mathbb{E}\Big{[}e^{-\rho T}V_{T}^{\alpha}+\int_{0}^{T}e^{-\rho s}f_{s}^{\alpha}ds\Big{]}.$$

$$\mathbb{E}[S_0^{\tilde{\alpha }}]=V^C$$

$$v_t(x,\bar{x})$$

$$d{K}_t={\dot{K}}_{t}dt,$$

$$d{Y}_t={\dot{Y}}_{t}dt+Z_t^{Y}d{W}_t^0,$$

$$d\mathbb{E}[S_t^{\alpha }]$$

$$\mathbb{E}[{\cal D}_{t}^{\alpha}]=\mathbb{E}\Big{[}-\rho v_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}])+\frac{d}{dt}\mathbb{E}\big{[}v_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}])\big{]}+f_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}],\alpha_{t})\Big{]},\;\;t\geq 0.$$

$$\mathbb{E}[{\mathcal{D}}_t^{\alpha }]\ge 0,t\ge 0,\forall \alpha \in \mathcal{A},$$

$$\mathbb{E}[{\mathcal{D}}_t^{\alpha }]$$

$$F(K_t,{\dot{K}}_t)=0,$$

$$H_t(K_t,{\Lambda }_t,Y_t,\mathbb{E}[Y_t],{\dot{Y}}_t)=0,$$

$${\hat{\alpha }}_t$$

$$\mathbb{E}[{\hat{X}}_t]$$

$$K=\gamma \frac{-(\rho -{\sigma }^2)+\sqrt{(\rho -{\sigma }^2{)}^2+\frac{4b^2\eta }{\gamma }}}{2b^2},\Lambda =\gamma \frac{-\rho +\sqrt{{\rho }^2+\frac{4b^2{\sigma }^{2}K}{\gamma }}}{2b^2}.$$

$${\hat{\alpha }}_t$$

$$\underset{t\to \infty }{\lim }\mathbb{E}[{\hat{X}}_t]=\frac{(1+\theta )\bar{P}-\frac{\rho c}{b}}{2{\sigma }^{2}K}=:\overline{{\hat{X}}_{\infty }}(\bar{P}),$$

$${\int }_0^t{e}^{-\frac{b^2\Lambda }{\gamma }(t-s)}{\int }_s^{\infty }{e}^{-(\rho +\frac{b^2\Lambda }{\gamma })(u-s)}\mathbb{E}[P_u]duds$$

Full text

JEL Classification: L94, C73, C61, O33, Q41, Q42.

MSC Classification: 91B42, 93E20, 91A15.

Key words: decarbonation, distributed generation, stochastic game, McKean-Vlasov.

1 Introduction

This paper analyses the interaction between centralised carbon emissive technologies and distributed intermittent non-emissive technologies. We start with the premise that distributed energy sources (mainly solar energy) give consumers an instrument that increases their bargaining power in their interaction with electric firms. This new deal in power systems entails a strategic interaction between these firms industry-wide and consumers now empowered with generation capacities. But this interaction is not of purely competitive type. Distributed generation is intermittent, highly dependent on weather conditions, and might not be enough at each instant to satisfy consumers’ electricity demand. To avoid extra costs to compensate for the intermittency of distributed generation (e.g. batteries), consumers might still want to have access to centralised generation as insurance.

The economic literature on the development of renewable energy has focused on the real values of renewable energy technologies (wind and solar) and in particular, on the externality cost they induce in electric systems, and on what should be the optimal share of renewable energy in a given power system. The main references on this thoroughly studied subject are Joskow (2011) [15], Borenstein (2012) [7], Hirth (2013) [12] and more recently by Gowrisankaran et al. (2016) [11]. Another topic is the existence or not of a capacity threshold where renewable energy is viable and no longer needs subsidies. Green and Léautier (2015) [17] address this question and find that achieving a low-carbon power system without permanent subsidy might not be possible because renewable energy drives electricity prices down as its market share increases. Another important issue is how the distribution tariffs should be adapted to take into account the new phenomenon of power injection at the distribution level (see [8] and references therein for a description of the metering problem). To our knowledge, our paper is the first to study the development of renewable energy as a strategic interaction situation between consumers and electricity firms.

We consider a representative consumer and a firm representing the electricity producers as a whole. Since our focus is on the effect of the distributed generation intermittency, we assume that the consumer’s demand is inelastic and constant. The consumer can invest in solar energy while the firm invests in power plants. Both perform their investments at a continuously controlled investment rate. We assume that the firm invests in dispatchable generation facilities (i.e. non-random) and sells its production to the consumer at a price he fixed. The dispatchable technology is carbon emissive and its generation is subject to a carbon tax. The carbon tax decreases as the distributed non-emissive technology share increases. The objective of the firm is to satisfy the electricity demand of the consumer at least total discounted expected costs. The firm’s cost function has three parts: the investment cost, the penalty for deviating from demand, and the revenue from selling electricity to the consumer. The firm decides the price at which to sell its energy to the consumer. This price is a random process. The consumer’s objective is also to minimise the total discounted expected costs to satisfy her demand, either by using her own random solar energy or by buying energy from the firm. The consumer has disutility from the intermittency of distributed energy. This disutility is quantified as a penalisation of the variance of the distributed energy. Further, the centralised energy produced by the firm is characterised by the existence of transmission costs that are assumed to be proportional to the energy price and that are paid by the consumer. Although the burden of transmission costs can be allocated by the transmission operator to the firm in some countries (e.g. in the United Kingdom), it is transferred to the consumer by the firm.

Although this model simplifies the complexities of the investment decisions in electricity generation, it captures a key new feature in today’s power systems and a new economic problem facing electric firms. If the firm decides to charge a high price for its energy to maximise its profit, then the consumer has the possibility to escape the firm’s market power by investing in distributed energy and generating her own electricity. However, charging too low a price to avoid this situation could impede the firm from recovering its investment costs. This trade-off is at the core of the interaction problem between the consumer and the firm. If the consumer has no other alternative than buying electricity from the firm, the equilibrium will be fixed by the standard theory of marginal cost pricing that depends on the demand elasticity of the consumer (see Boiteux (1956,1960) [6, 5]). Here, the availability of the option to invest in distributed generation changes the respective bargaining powers of the consumer and the firm. The consumer can escape the firm’s market power at the expense of investing in a costly alternative. Thus, the firm might be tempted to offer suboptimal prices to the consumer to prevent her from investing in local generation.

This paper’s first contribution is to give the optimal investment strategy for both players without any assumptions in the model on the energy price. The optimal investment strategies are solutions of stochastic control problems with random coefficients. Moreover, in the case of the consumer’s problem, the penalisation of the variance leads to a new type of stochastic control problem, called McKean-Vlasov control problems. Both problems admit closed-form solutions depending on the price process of the centralised energy. Using only the hypothesis that the price process admits a long-term stationary value, we derive the optimal long-term behaviour of each player. In the long-term, the consumer’s cumulated distributed capacity will reach a positive limit, as long as the long-term price of centralised energy plus transmission cost is greater than the annuity of distributed energy (total discounted investment costs). In particular, the consumer decides her long-term cumulated capacity based on the centralised energy price compared to the distributed energy cost and her aversion to intermittency. Centralised energy costs which are too high can result in investment that exceeds consumers’ demand. The firm’s cumulated capacity will be equal to the residual demand of the consumer (demand minus installed distributed capacity). The independence of these results with respect to the price models guarantees their robustness.

Our second contribution is to provide the conditions for the existence and characterisation of a long-term Pareto optimum between the distributed and centralised generations. We compute this optimum and give conditions under which it can be implemented by setting a regulated price for centralised energy. Once again, we obtain these results by considering the centralised energy price as a random parameter, that is without making any model hypothesis.

We find that if the price of the centralised energy admits a stationary distribution, then the levels of installed centralised and distributed generation determined by the social planner admits stationary limits. The long-run level of centralised generation is equal to the residual demand of the consumer while the level of the distributed generation nontrivially depends on the costs of both the distributed and centralised technology costs. However, some limiting cases are worth noting. In the limiting case where there are transmission costs and no carbon tax, distributed generation admits a long-term positive optimum value. This value increases with the long-run value of the centralised energy price, decreases with the distributed investment cost annuity and with the intermittency measured by the variance of the distributed generation. This long-run optimum is achieved at an exponential rate. When there is neither a carbon tax nor transmission costs, there is no investment in distributed generation in the long-run, whatever the respective investment costs of both technologies: The social planner who is risk-averse to intermittency will invest in high cost dispatchable technology rather than in low cost intermittent technology. This unexpected result is not a consequence of the variance penalisation, but is actually induced by the (non-random) demand satisfaction objective. Indeed, the intermittency of distributed capacity always results in a positive cost whereas dispatchable centralised technology can cancel this cost by perfectly matching the demand.

Moreover, we can implement a stationary Pareto optimum price. By fixing the long-run value of the centralised energy price, the social planner can achieve the Pareto optimal long-run value of distributed generation, under certain explicit conditions. This asymptotic Pareto optimum price can be interpreted as a long-term contract offered to the firm to guarantee its return on investments. We find that as soon as the distributed energy per unit cost has reached a certain level, the free interaction between the technologies will lead to an equilibrium in the long-run. Thus, for a given target level of distributed energy, there is a distributed energy cost under which subsidies are no longer required to achieve an asymptotic Pareto optimum. This result somehow tempers the above cited conclusion of Green and Léautier (2015) [17]. We extend our model to the situation of multiple consumers to check whether or not the results depend on the reduction of the interaction between technologies to a single representative agent. We find that the same general results still apply, up to a correcting term given by the return to scale provided by the cost reduction induced with a large number of players. Moreover, some important limiting cases admit further analysis. In particular, we find that the implementation of the asymptotic Pareto optimum relies on the existence of a positive carbon tax. Contrary to the case where the social planner could enforce the long-run equilibrium by choosing himself the level of investment, the use of a price requires a positive carbon tax to ensure a positive distributed generation. In this situation, the transmission costs are no longer enough to sustain the development of distributed energy sources on the long-run.

Our third contribution is the study of a the Stackelberg equilibrium where the firm is the leader. This modelling represents an unregulated market interaction between the two players. In this game, the firm chooses both its installed capacity and its selling price and then, the consumer chooses her level of installed solar panels. Because the firm determines its price and the consumer’s demand is inelastic, a Nash equilibrium is not a realistic consideration while the Stackelberg equilibrium captures the asymmetry of the two players. We provide an explicit condition under which there is an asymptotic Stackelberg equilibrium that we compute. We find that it leads to significant differences than in the Pareto optimum such as a higher price for the centralised energy and a significant over-investment in distributed energy.

As a result, this comparative analysis of different interactions between the firm and the consumer shows that if there is not a large carbon emission price, it is unlikely that we can achieve a social welfare Pareto optimum either by letting the players freely interact or by a public intervention limited to a regulated centralised energy price.

The outline of the paper is as follows. Section 2 presents our model and its hypotheses. In Section 3 we describe the optimal behaviour of each player (firm, consumer, social planner). Section 4 provides the different possible equilibria. Section 5 provides extensions with a large number of firms and consumers, and we conclude in Section 6.

2 Model and problem formulation

We consider an interaction model on electricity market that involves the following actors: there is a firm (he) which provides electricity with centralised fully dispatchable generation (typically coal- or gas-fired plants) to a representative consumer who can also produce her own electricity by investing in renewable energy, typically by purchasing solar panels. The consumer’s demand for electricity is then satisfied both with the distributed generation of the solar panels and with the centralised generation bought at the price set by the firm. The problem for the (representative) consumer is then to decide how many solar panels she must install. On the other hand, the firm has to satisfy the residual demand of the consumer with its generation plants while minimising his production costs and taking into account carbon taxes due to emission of thermal energy. Further, we consider a third player in this game, namely a social planner, who is typically looking for a price of the centralised generation, which is Pareto-optimal for the firm and the consumer.

We now formulate mathematically the model and the optimisation problem for each actor in this three-player game. We fix some probability space $(\Omega,{\cal F},\mathbb{P})$ equipped with a filtration $\mathbb{F}$ $=$ $({\cal F}_{t})_{t\geq 0}$ satisfying the usual conditions and supporting two independent standard Brownian motions $W$ and $W^{0}$. The consumer’s cumulative net available capacity of distributed generation at any time $t$ is denoted by $X_{t}^{\alpha}$ and is governed by:

[TABLE]

where the $\mathbb{F}$-adapted control process $\alpha$ $=$ $(\alpha_{t})_{t\geq 0}$ represents the solar panel installation rate ($\alpha_{t}$ $>$ [math] means that the consumer buys the at time $t$, while $\alpha_{t}$ $<$ [math] means that the consumer sells them), $b>0$ is the load factor of the solar panels. Typically, the installation rate $\alpha_{t}$ is measured in megawatt per unit of time, and $b$ is a dimensionless parameter. The parameter $\sigma$ $>$ [math] is a volatility coefficient related to the uncertainty in the net electricity available capacity of photovoltaics (its net capacity depends on weather conditions), and the proportional term w.r.t. $X_{t}^{\alpha}$ in the noise means that the uncertainty increases with the level of generation. The volatility term of the intermittent generation technology has been chosen to reflect that the more the consumer relies on this technology, the more he has to cope with uncertainty. Our model is not a geographical model where the total generation of intermittent electricity sources could result in a smoother generation by an aggregation effect. The form of the volatility captures the idea that the consumer faces a risk of a low production whatever the level of installed capacity. This point is a concern of many transmission system operators managing electric network with a high level of renewable energy penetration (see German Energy Regulator Annual Report on the constant increase of redispatching cost to cope with intermittent energy sources [9], Fig. 64, p. 173, and Fig. 65, p. 174). The demand process of the (representative) consumer, denoted by $D$, is an exogenous factor in our model, and is assumed to be constant. We justify this hypothesis in the following way: The problem of the consumer as well as the social planner is to make an arbitrage between non-emissive intermittent technologies and emissive dispatchable technologies. Uncertainty on the demand only complicates the computations without bringing more insights on the choices of the consumer or the social planner. Another way of justifying this hypothesis is to consider that the volatility $\sigma$ is the net uncertainty generated by the distributed generation plus demand. In order to satisfy the consumer’s residual demand (i.e. the electricity demand not fulfilled by the photovoltaic) at any time $t$, the consumer purchases electricity from the firm at price $P_{t}$ as chosen by the firm. The price process $(P_{t})_{t\geq 0}$ should not be considered as a real-time pricing process, typically the day-ahead hourly spot price that is quoted on electricity markets. It is more pertinent to see this price as the annual tariff that appears on consumers’ electricity bill because the price is announced in advanced to the consumer by the firm in the case of a Stackelberg equilibrium. For this reason, the price process $P$ $=$ $(P_{t})_{t\geq 0}$ is adapted with respect to the filtration $\mathbb{F}^{0}$ $=$ $({\cal F}_{t}^{0})_{t\geq 0}$ generated by $W^{0}$. In other words, we assume that the price of the centralised generation is not affected by local issues like the weather, but only by macroeconomic and market factors. However, we do not make a priori model assumptions on the price process since our ultimate goal is to endogenously derive the price by equilibrium arguments. Over an infinite horizon, the expected total discounted cost of the consumer for an installation rate $\alpha$ is:

[TABLE]

where $\rho$ $>$ [math] is some discount factor which we assume is large enough for ensuring the well-posedness of our problem. The first term $C_{1}(\alpha_{t})$ represents the instantaneous installation cost for photovoltaics where $C_{1}$ a convex function on $\mathbb{R}$, $C_{1}(0)$ $=$ [math] which we assume in the sequel to be in the quadratic form:

[TABLE]

where $c$ $\geq$ [math] is the per unit constant cost of distributed power, and $\gamma$ $>$ [math] is the market impact factor.

The hypothesis of a constant cost $c$ of distributed energy is challenging because in real life, an important decrease in per unit cost of renewable energy sources in the last 20 years has occurred because of both economies of scale, learning by doing and technological progress (see Reichelstein et al. [19] and Rubin et al. [21]). Nevertheless, the study of the equilibrium between centralised and distributed generation when consumer has access to a technology at constant per unit cost $c$ already reveals a lot about their interaction. Furthermore, we present in Section 5 an extension where we relax the hypothesis of a constant per unit cost of solar panels to take into account an increasing return to scale. The quadratic term of the cost function captures the fact that the marginal cost of 1 MW of solar panels will increase if the buying rate is too fast. If the representative consumer wants to increase her installation rate, solar panels production firms need to buy resources at higher prices and thus increase their price. This phenomenon was observed during the renewable energy boom (see [22] p. 51, Fig. 32). Further, the small negative $a$ corresponds to a situation where the consumer sells back her solar panels. In this sense, the investment problem for the consumer has a reversibility property. The second term $(1+\theta)P_{t}(D-X_{t}^{\alpha})$ is the (algebraic) cost for satisfying the residual demand with the centralised electricity. The parameter $\theta$ represents the costs that add up to the energy price $P_{t}$ on the consumer’s electricity bill due to the necessary transmission and distribution infrastructure required to bring centralised generation to her household. In this regard, $\theta$ measures the extra cost induced by centralised generation compared to distributed generation. It is not small. Transmission costs can represent as much as two-thirds of the consumer’s bill, leading to an $\theta=2$. In this model, we assume that $\theta$ is exogenous and fixed and leave for future research the potential strategic effect it might generate between distributed energy adopters and non adopters (see [8] for a description of the effect of metering on the repartition of network costs between different type of consumers). The distributed generation might potentially exceed the demand which would correspond to a situation where the consumer sells the excess generation to the firm. But since our model is closed (there is no other consumer to whom this excess generation could be sold), we place conditions on the definition of an equilibrium that excludes this case (for this reason, we keep the cost functions unchanged when the distributed generation exceeds the demand). The third term represents the preference of the consumer for low-variance policies (quantified by the penalty parameter $\eta$ $>$ [math]) so as to get stable flux in the distributed generation, and thus be less sensitive to the market risk of a centralised electricity price. The consumer has to decide how much distributed generation she has to self-produce from the solar panels, and her goal is to minimise over the installation rate control $\alpha$ the expected total discounted cost $J_{1}(\alpha)$ in (2.2). The solution to this problem will be studied in Section 3.1. Notice that the corresponding optimal self-production $\hat{X}$ $=$ $\hat{X}(P)$ for the consumer depends on the thermal price $P$.

On the other hand, knowing the (optimal) distributed generation of the consumer, the firm has to choose its centralised generation capacity for satisfying the (residual) demand of the consumer while taking into account the operational costs, the gain from the sale of electricity to the consumer, and carbon taxes for carbon emission. We denote $Q^{\nu}$ as the firm’s cumulative production of centralised generation given by the dynamics

[TABLE]

where $\nu$ $=$ $(\nu_{t})_{t\geq 0}$ is an $\mathbb{F}$-adapted real-valued process, which represents the production installation rate. However, compared to the generation from distributed generation, there is no uncertainty in centralised generation (no stochastic noise on the evolution of $Q^{\nu}$) which captures the fact that centralised generation is fully dispatchable. Given a level of distributed generation capacity $X^{\alpha}$, the aim of the energy firm is to minimise over its production rate $\nu$ of centralised generation capacity the expected total discounted cost:

[TABLE]

The first term in (2.4) represents the production cost, and therefore we consider a quadratic cost function of

[TABLE]

for some positive constant $h$ $>$ [math]. Contrary to the investment cost function of the consumer, there is no linear term in the firm’s investment cost function. We do not add this linear term because the investment in centralised generation capacity is irreversible (at least, much more irreversible than investing in distributed generation). Increasing or decreasing capacity induces important costs to the firm. The second term is a quadratic penalisation, which constrains (more and less depending on the size of the parameter $\lambda>0$) the firm to fit the residual demand of the consumer with its generation. In electric systems, generation is always equal to consumption. Thus, the commitment of the producer to satisfy the consumer’s electricity demand translates into an almost sure constraint stating $Q^{\nu}_{t}=D-X^{\alpha}_{t}$. But, because $Q^{\nu}$ and $X^{\alpha}$ are controlled processes, it is not possible to ensure this equality almost surely. A natural way to circumvent this difficult in a highly stylised model of power system is to resort to a penalisation of the demand satisfaction constraint. The term $\lambda(D-X^{\alpha}_{t}-Q^{\nu}_{t})^{2}$ can be interpreted as a penalisation of the demand satisfaction constraint in an augmented Lagrangian, a technique which is common in unit commitment model to find an optimal dispatch [20].

The third term is the gain from the sale of centralised generation to the consumer at price $P_{t}$. Although the consumer pays the centralised energy $(1+\theta)P_{t}$, the firm only receives $P_{t}$. The fourth term represents a carbon tax for the emission of thermal energy, where we set

[TABLE]

for some constants [math] $\leq$ $\pi_{0}$ $<$ $\pi_{1}$, as the amount of carbon tax to be paid for one unit of energy sold at time $t$. Since the demand is expected to fit the total production from distributed and centralised generations, $D$ $\simeq$ $X_{t}^{\alpha}+Q_{t}^{\nu}$, then $\pi_{t}^{\alpha,\nu}$ is approximately a convex combination of $\pi_{0}$ and $\pi_{1}$ and is valued in $[\pi_{0},\pi_{1}]$. It increases with the level of emissive centralised generation, is close to $\pi_{1}$ when the demand is mostly satisfied by thermal production, and is close to $\pi_{0}$ when the distributed generation covers almost all the demand. We choose the parameter $\pi_{0}$ close to the smallest value of observed carbon emission allowances price ($\pi_{0}\approx 2$ euro per metric ton) whereas we choose $\pi_{1}$ close to the penalty value of firm’s missing their emission target ($\pi_{1}\approx 100$ per metric ton). Making both $\pi_{0}$ and $\pi_{1}$ equal to zero allows to study the case of an equilibrium with centralised non-emissive technology. The solution to this problem will be studied in Section 3.2. Note that the corresponding optimal centralised capacity $\hat{Q}$ $=$ $\hat{Q}(X^{\alpha})$ for the firm depends on the distributed generation $X^{\alpha}$ of the consumer.

Besides the consumer and the centralised firm, we consider a social planner, whose aim is to minimise over the investment rate pair $(\alpha,\nu)$ the sum of expected costs from the distributed and centralised generations:

[TABLE]

This problem will be studied in Section 3.3. The corresponding optimal distributed and centralised generation capacities are denoted by $(X^{*},Q^{*})$ $=$ $(X^{*}(P),Q^{*}(P))$. The cost function of the social planner still depends on the centralised energy price $P$. The energy price is not a pure transfer price from the consumer to the firm. It entails a social cost due to the need for the transmission infrastructure. The chief goal of the social planner is to endogenise the electricity market price $P$ by equilibrium arguments. We first consider the concept of Pareto-optimality for the consumer and the energy firm, which is such that the optimal distributed generation capacity for the consumer and the optimal centralised generation capacity for the firm coincides with the solution derived from the social planner’s problem. In other words, the mathematical problem is to determine a price process $P^{*}$ such that $\hat{X}(P^{*})$ $=$ $X^{*}(P^{*})$ and $\hat{Q}(\hat{X}(P^{*}))$ $=$ $Q^{*}(P^{*})$. We also consider the concept of equilibrium price when focusing on a two-player game between the firm and the consumer (hence without the social planner). In this context, we view the firm as a leader which offers a price to the consumer (the follower), and the endogenous price results from a Stackelberg equilibrium where the leader optimises the expected total cost through the price and its investment rate given the best reaction effort (the distributed generation) of the follower. These two equilibrium issues will be discussed and addressed in Section 4.

3 Optimal behaviour of the players

In this section, we address the optimisation problems for each player (consumer, firm and social planner) and determine their optimal generation investment rate explicitly.

3.1 The consumer’s problem

From the expression (2.2) of the expected total discounted cost of the consumer, and removing the term $P_{t}D$ that does not depend on the installation rate control $\alpha$, the consumer’s problem is written as

[TABLE]

where $\{f_{t}(x,\bar{x},a),(x,\bar{x},a)\in\mathbb{R}^{3},t\geq 0\}$ is the $\mathbb{F}^{0}$-adapted random field defined by

[TABLE]

Here ${\cal A}$ is the set of admissible controls defined as the set of $\mathbb{F}$-adapted real-valued processes $\alpha$ s.t. $\mathbb{E}[\int_{0}^{\infty}e^{-\rho t}|\alpha_{t}|^{2}dt]$ $<$ $\infty$. We assume that the positive discount factor $\rho$ satisfies:

[TABLE]

which implies from Itô’s formula on (2.1), Young’s inequality and Gronwall’s lemma that

[TABLE]

for $\epsilon>0$ small enough. We also assume that the price process satisfies the square-integrability condition

[TABLE]

which ensures by (3.4) that problem (3.1) is well-defined. Problem (3.1) is a nonstandard stochastic control problem due to the variance term, which induces a priori time inconsistency. It belongs to the class of so-called McKean-Vlasov (MKV) control problems where the running cost $f_{t}$ in (3.2) involves the law of the state process $X^{\alpha}$ in a nonlinear dependence coming from the square of its mean. McKean-Vlasov control problems have received a surge of interest in the last few years in connection with the mean-field game theory initiated by Lasry and Lions, and we refer for example to [4], [10], and [18] for recent works on this topic. Problem (3.1) falls more specifically into the class of linear-quadratic MKV control problems with random coefficients that have been recently studied in [3]. The method of resolution of (3.1) is based on a mean version of martingale optimality principle in dynamic programming leading to the following verification argument.

Lemma 3.1** (Optimality principle).**

Let $\{V^{\alpha},\alpha\in{\cal A}\}$ be a family of $\mathbb{F}$-adapted processes in the form $V_{t}^{\alpha}$ $=$ $v_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}])$, $t\geq 0$, for some random field $\mathbb{F}$-adapted process $\{v_{t}(x,\bar{x})$, $(x,\bar{x})\in\mathbb{R}^{2}$, $t\geq 0\}$ satisfying the following conditions:

• (i)

there exist a positive constant $C$ and a $\mathbb{F}$-adapted process $(I_{t})_{t\geq 0}$ such that $\mathbb{E}[e^{-\rho T}I_{T}]$ $\rightarrow$ [math] as $T\to\infty$ and that $|v_{t}(x,\bar{x})|\leq C(I_{t}+|x|^{2}+|\bar{x}|^{2})$, for $t\geq 0$ and $(x,\bar{x})\in\mathbb{R}^{2}$;

• (ii)

for all $\alpha$ $\in$ ${\cal A}$, the map $t$ $\in$ $\mathbb{R}_{+}$ $\mapsto$ $\mathbb{E}[S_{t}^{\alpha}]$, with $S_{t}^{\alpha}$ $=$ $e^{-\rho t}V_{t}^{\alpha}+\int_{0}^{t}e^{-\rho s}f_{s}(X_{s}^{\alpha},\mathbb{E}[X_{s}^{\alpha}],\alpha_{s})ds$, is nondecreasing;

• (iii)

there exists some $\hat{\alpha}$ $\in$ ${\cal A}$ such that the map $t$ $\in$ $\mathbb{R}_{+}$ $\mapsto$ $\mathbb{E}[S_{t}^{\hat{\alpha}}]$ is constant.

Then $\hat{\alpha}$ is an optimal control for problem (3.1) and $V^{C}$ $=$ $\mathbb{E}[v_{0}(x_{0},x_{0})]$. Moreover, any other optimal control satisfies the condition (iii).

Proof. Let $\alpha\in\mathcal{A}$ and $T>0$. Setting $f_{t}^{\alpha}$ $:=$ $f_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}],\alpha_{t})$, from (ii) we get

[TABLE]

By (i) and (3.4) we have $\mathbb{E}[e^{-\rho T}|V_{T}^{\alpha}|]\to 0$ as $T\to\infty$; hence, by sending $T$ to infinity into (3.6) and by using the dominated convergence theorem ($f_{t}$ satisfies a quadratic growth condition), we have $\mathbb{E}[v_{0}(x_{0},x_{0})]\leq\mathbb{E}[\int_{0}^{\infty}e^{-\rho s}f_{s}^{\alpha}ds]$ and then $\mathbb{E}[v_{0}(x_{0},x_{0})]$ $\leq$ $V^{C}$, since $\alpha$ is arbitrary in ${\cal A}$. Similarly, by (iii) we obtain $\mathbb{E}[v_{0}(x_{0},x_{0})]$ $=$ $\mathbb{E}[\int_{0}^{\infty}e^{-\rho s}f_{s}^{\hat{\alpha}}dt]$ $\geq$ $V^{C}$, so that $\hat{\alpha}$ is an optimal control and $V^{C}$ $=$ $\mathbb{E}[v_{0}(x_{0},x_{0})]$. Suppose now that $\tilde{\alpha}$ $\in$ ${\cal A}$ is another optimal control. For all $T>0$ we have

[TABLE]

so that, by sending $T$ to infinity, we get $\mathbb{E}[S_{0}^{\tilde{\alpha}}]$ $=$ $\lim_{T\rightarrow\infty}\mathbb{E}[S_{T}^{\tilde{\alpha}}]$; since the map $t$ $\mapsto$ $\mathbb{E}[S_{t}^{\tilde{\alpha}}]$ is nondecreasing, this shows that this map is actually constant. $\Box$

Remark 3.1**.**

The verification conditions (ii) and (iii) in Lemma 3.1 are weaker than the usual conditions for martingale optimality principle in stochastic control (see [16]), which require that $S^{\alpha}$ is a supermartingale for all $\alpha$, and a martingale for some $\hat{\alpha}$. In the case of a standard (without McKean-Vlasov dependence) stochastic control problem, one looks for a family of processes $V^{\alpha}$ in the form $V_{t}^{\alpha}$ $=$ $v_{t}(X_{t}^{\alpha})$, and the classical martingale optimality principle is used in practice by applying Itô’s formula to $S_{t}^{\alpha}$ and then deriving a dynamic programming Hamilton-Jacobi-Bellman equation (or backward stochastic differential equation in the case of random coefficients) for the (random) value function $v_{t}(x)$ from the non-positivity of the drift of $S^{\alpha}$ for any $\alpha$ (supermartingale condition), and the cancellation of the drift of $S^{\hat{\alpha}}$ (martingale condition). In our McKean-Vlasov framework, this martingale optimality principle cannot be used in practice for finding the solution to the control problem, and we instead exploit the weaker version in Lemma 3.1 where the optimality principle is formulated on the mean of $S^{\alpha}$. This is the purpose of the next paragraph. * $\Box$*

3.1.1 Optimal distributed generation

We explain in this paragraph how to exploit the mean martingale optimality principle in Lemma 3.1 for solving problem 3.1. The technical details and complete rigorous derivation are postponed in Appendix A.1. Given the linear-quadratic structure of our MKV control problem, we look for a candidate random value function $v_{t}(x,\bar{x})$ in the form:

[TABLE]

for some $\mathbb{F}$-adapted processes $K$, $\Lambda$, $Y$ and $R$ to be determined. Because the randomness in the coefficients of the consumer’s problem comes only from the $\mathbb{F}^{0}$-adapted price process $P$ in the linear term in $f_{t}$, we search for deterministic $K$, $\Lambda$, and $\mathbb{F}^{0}$-adapted processes $Y$ and $R$, hence in the evolution form (recall that $\mathbb{F}^{0}$ is the filtration of the Brownian motion $W^{0}$):

[TABLE]

for some deterministic processes $\dot{K}$, $\dot{\Lambda}$, and $\mathbb{F}^{0}$-adapted processes $\dot{Y}$, $\dot{\Gamma}$, $\dot{R}$, $Z^{Y}$, $Z^{R}$. Next, applying Itô’s formula to $S_{t}^{\alpha}$ $=$ $e^{-\rho t}v_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}])$ $+$ $\int_{0}^{t}e^{-\rho s}f_{s}(X_{s}^{\alpha},\mathbb{E}[X_{s}^{\alpha}],\alpha_{s})ds$, for $\alpha$ $\in$ ${\cal A}$, and taking the expectation, we get

[TABLE]

for some $\mathbb{F}$-adapted processes ${\cal D}^{\alpha}$ with

[TABLE]

According to Lemma 3.1, we have to determine $K$, $\Lambda$, $Y$, $\Gamma$, and $R$ so that

[TABLE]

which ensures that the mean optimality principle conditions (ii) and (iii) are satisfied, and then $\hat{\alpha}$ will be an optimal control. For this, we apply Itô’s formula to $v_{t}(X_{t}^{\alpha},\mathbb{E}[X_{t}^{\alpha}])$ recalling the quadratic form of $v_{t}(\cdot,\cdot)$, the dynamics of $X^{\alpha}$ in (2.1), hence of its mean: $d\mathbb{E}[X_{t}^{\alpha}]$ $=$ $b\mathbb{E}[\alpha_{t}]dt$, and after some calculations together with square completion, we obtain:

[TABLE]

where $A$, $F$, $G$, $H$, and $M$ are explicit functions of their arguments (see Appendix (A.1)). It follows that condition (3.9) is realised whenever the following equations are satisfied

[TABLE]

and

[TABLE]

The explicit resolution of these equations leads to our first main result which is a closed-form expression of the optimal distributed generation production for the consumer.

Theorem 3.1**.**

Let (3.3) and (3.5) hold. The optimal solar panel installation rate is given by

[TABLE]

with the optimal cumulative production of distributed generation $\hat{X}$ given in mean by

[TABLE]

where the positive constants $K$ $>$ $\Lambda$ are equal to

[TABLE]

Interpretation and comments. The expression (3.13) of the optimal solar panels installation rate consists in several explicit terms that have the following interpretation: the first term shows the mean-reversion of the production towards its mean due to the variance penalization in the criterion, and we notice that the speed of mean-reversion, proportional to $K$, increases with the penalty parameter $\eta$. The second term is related to the generation uncertainty of the solar panels, and decreases the installation rate all the more so as the current level of generation (on average) and its volatility $\sigma$ are high. The third term, which is negative, expresses the obvious feature that the higher is the fixed cost $c$ on solar panels installation, the smaller is the optimal rate of production. Finally, the fourth and fifth terms, depending on the electricity price, quantify the natural intuition that the higher is the expectation on the future price, the higher is the incentive to invest in photovoltaics for self-generation.

In particular, if we assume that the price $P_{s}$ is constant equal to $P$, the expression (3.13) takes the following form:

[TABLE]

The last term for the optimal investment rate is a constant rate of installation whose sign depends on the relative value of buying centralised energy at price $(1+\theta)P$ compared to the annuity of investment in distributed energy $\frac{\rho c}{b}$. If centralised energy is more costly than distributed energy, then the consumer installs distributed energy at a constant rate.

3.1.2 Long-term self-generation

We address the long-term behaviour of the optimal distributed generation when the price process has a stationary level, that is, $\mathbb{E}[P_{t}]$ $\rightarrow$ $\bar{P}$ for some positive constant $\bar{P}$ $>$ [math], as $t$ goes to infinity. This stationarity occurs for example when the price process is a martingale, in which case $\bar{P}$ $=$ $P_{0}$, or when it is mean-reverting (following e.g. an Ornstein-Uhlenbeck or a Cox-Ingersoll-Ross process) with a stationary level of $\bar{P}$.

Proposition 3.1**.**

Let (3.3) and (3.5) hold and assume that there exists $\bar{P}$ $>$ [math] s.t. $\lim_{t\rightarrow\infty}\mathbb{E}[P_{t}]$ $=$ $\bar{P}$. Then, the optimal cumulative production of distributed generation admits a stationary level:

[TABLE]

Proof. The first limit follows immediately from (3.14) by noting that

[TABLE]

by the dominated convergence theorem, and recalling that $\Lambda(\rho\gamma+b\Lambda^{2})=\gamma\sigma^{2}K$. As for the second limit, notice that $\mathbb{E}[\hat{X}_{t}]-x_{0}=b\int_{0}^{t}\mathbb{E}[\hat{\alpha}_{s}]ds$, so that $\int_{0}^{\infty}\mathbb{E}[\hat{\alpha}_{s}]ds$ is finite, which implies $\lim_{t\rightarrow\infty}\mathbb{E}[\hat{\alpha}_{t}]=0$. $\Box$

Interpretation and comments. The spread $(1+\theta)\bar{P}-\frac{\rho c}{b}$ is the difference for the consumer between the cost of centralised energy and distributed energy ($\rho c/b$ is the annuity in €/MW/year of distributed energy). The consumer will invest in distributed energy only if this difference is positive. The consumer’s investment will be proportional to this spread and inversely proportional to the volatility of the distributed generation (for reasonable parameter values, it is seen that $K$ weakly depends on the volatility). The case of a negative spread leads to a spurious negative long-term installed capacity because we allow the consumer to sell her generation to the firm. However, this long-term behaviour does not depend on the consumer’s demand. Since the consumer is not compelled to make her generation close to her demand, the investment choice is purely driven by the relative prices of centralised versus distributed generation. Thus, high centralised energy prices might induce an over-investment in distributed generation.

Numerical illustration. To fix an idea of the order of magnitude of possible investment in distributed generation, consider the following values based on common order of magnitudes for the parameter of the model: $\rho=0.1$ per annum, $\sigma=0.3$ per annum, $b=0.15$, a unitary cost $c=3000$ €/kW, a price of centralised energy of $\bar{P}$ $=$ 80 €/MWh and a transmission cost $\theta=2$. The market impact parameter is taken to be small at a level of $\gamma=1$ €/MW2/year to avoid overestimating its effect as well as the cost of variance of distributed generation that we take to be $\eta=8760\times 10$ €/MW2/year. The resulting spread price $(1+\theta)\bar{P}-\frac{\rho c}{b}$ expressed in €/MWh equals 11 €/MWh and the consumer will invest $\overline{\hat{X}_{\infty}}(\bar{P})$ $=$ $288$ MW in solar energy. Dividing the market impact parameter $\gamma$ by 100 results in an installed distributed generation multiplied by 10. Multiplying the penalty parameter of the variance $\eta$ by 100 results in an installed distributed generation divided by 10.

3.1.3 Mean-reverting price process

We illustrate numerically the above results on the optimal distributed generation for the consumer when the price process is mean-reverting. Hence we consider dynamics in the form

[TABLE]

for some constants $\kappa$ $\geq$ [math], $\bar{P}$ $>$ [math], and where the (possibly random) volatility $\xi_{t}$ of the price process is a $\mathbb{F}^{0}$-adapted positive process such that

[TABLE]

This modelling includes the cases where $P$ is a martingale ($\kappa$ $=$ [math]), $P$ is an Ornstein-Uhlenbeck process ($\xi_{t}$ $=$ $\xi$ positive constant), $P$ is governed by a Cox-Ingersoll-Ross process ($\xi_{t}$ $=$ $\xi\sqrt{P_{t}}$ for some $\xi$ $>$ [math] that satisfies $2\kappa\bar{P}$ $>$ $\xi^{2}$). For such a mean-reverting process, we have

[TABLE]

the optimal solar panel installation rate in Theorem 3.1 can be written explicitly in terms of the current self-production level $\hat{X}_{t}$, the initial level $x_{0}$, the current price $P_{t}$, the initial value $P_{0}$, and the stationary level $\bar{P}$. If $b^{2}\Lambda-\kappa\gamma\neq 0$, then the optimal control is

[TABLE]

and the average self-production level is:

[TABLE]

If $b^{2}\Lambda-\kappa\gamma=0$, then the optimal control is

[TABLE]

and the average self-production level is:

[TABLE]

Further, $\mathbb{E}[\hat{X}_{t}]$ converges exponentially fast to its stationary level $\bar{x}(\bar{P})$. We plot in Figure 2 a sample trajectory of the optimal control $\hat{\alpha}$, of the optimal production levels $\hat{X}$, and of the price process (an Ornstein-Uhlenbeck model for $P$). We also illustrate the convergence of the optimal production to its stationary level in Figure 2. Further, we illustrate the effect of the variance penalty parameter $\eta$ by plotting 50 trajectories of the optimal cumulative self-production $\hat{X}$ in Figures 4 and 4.

3.2 The firm’s problem

We start from the expression (2.4) of the expected total discounted cost of the firm and remove the terms that are not affected by the centralised production. Thus, given a distributed generation production $X^{\alpha}$, we wrtite the firm’s problem as

[TABLE]

where $\{\ell_{t}(q,u),(q,u)\in\mathbb{R}^{2},t\geq 0\}$ is the $\mathbb{F}^{0}$-adapted random field defined by

[TABLE]

Here ${\cal V}$ is the set of $\mathbb{F}$-adapted real-valued processes $\nu$ s.t. $\mathbb{E}[\int_{0}^{\infty}e^{-\rho t}|\nu_{t}|^{2}dt]$ $<$ $\infty$, which implies that $\mathbb{E}[\int_{0}^{\infty}e^{-\rho t}|Q_{t}^{\nu}|^{2}dt]$ $<$ $\infty$. Because $\mathbb{E}[\int_{0}^{\infty}e^{-\rho t}|X_{t}^{\alpha}|^{2}dt]$ $<$ $\infty$ for $\alpha$ $\in$ ${\cal A}$, the problem (3.17) is well-defined. Problem (3.17) fits into the class of linear-quadratic stochastic control problems with random coefficients and on infinite horizon. We can solve the problem by following the same martingale optimality principle as for the consumer’s problem, although without the McKean-Vlasov dependence (see Remark 3.1). We provide a closed-form expression for the optimal centralised generation production of the firm, as well as its long-term behaviour. The proof is reported in Appendix A.2.

Theorem 3.2**.**

Given a distributed generation $X^{\alpha}$, the optimal rate $\hat{\nu}(X^{\alpha})$ for centralised generation is given by

[TABLE]

with the optimal cumulative production $\hat{Q}(X^{\alpha})$ given in mean by

[TABLE]

and where $\tilde{K}$ is the positive constant equal to

[TABLE]

Moreover, when the distributed production process has a stationary level, that is $\lim_{t\rightarrow\infty}\mathbb{E}[X^{\alpha}_{t}]$ $=$ $\bar{X}_{\infty}^{\alpha}$ for some $\bar{X}_{\infty}^{\alpha}$ $>$ [math], then the optimal cumulative production of centralised generation also admits a stationary level:

[TABLE]

In particular, if $\lim_{t\rightarrow\infty}\mathbb{E}[P_{t}]$ $=$ $\bar{P}$ exists and we consider the optimal distributed generation $\hat{X}$ $=$ $X^{\hat{\alpha}}$, then the limit of the centralised production $\hat{Q}$ $=$ $\hat{Q}(\hat{X})$ reads as

[TABLE]

where $\overline{\hat{X}_{\infty}}$ is defined in (3.16).

Interpretation and comments. In the long-term the firm invests the fraction $1-\pi_{1}/(2\lambda D)$ of the residual demand of the consumer. Nevertheless, the ratio $\pi_{1}/(2\lambda D)$ is close to zero. Indeed, as pointed in Section 2, if we consider that centralised energy is produced by coal-fired plants (1 MWh will generate 1 tonne of carbon emission), then $\pi_{1}\approx 100$ €/MWh. Moreover the demand is of the order of magnitude of 50000 MW for a country like France or a state like California. The parameter $\lambda$ is also significantly large to compel the firm to satisfy consumer demand. Thus, it results that $\pi_{1}/(2\lambda D)$ is very small compared to 1 and that the firm invests a capacity equal to consumer’s residual demand.

3.3 The social planner’s problem

The social planner’s problem, which consists in minimising the expected total costs of the consumer and the firm, is written as

[TABLE]

where $\{g_{t}(x,\bar{x},q,a,u),(x,\bar{x},q,a,u)\in\mathbb{R}^{5},t\geq 0\}$ is the $\mathbb{F}^{0}$-adapted random field defined by

[TABLE]

Problem (3.24) belongs to the class of linear quadratic McKean-Vlasov control problems in dimension 2, and is also solved by the same martingale optimality principle as described in Section 3.1. The next result, whose proof is reported again in Appendix A.3, gives the closed-form expression for the optimal distributed and centralised generation determined by the social planner.

Theorem 3.3**.**

Let (3.3) and (3.5) hold and assume that

[TABLE]

Then, the optimal distributed and centralised generation for the social planner are given by

[TABLE]

where $\{K^{ij}\}_{1\leq i,j\leq 2},\{\Lambda^{ij}\}_{1\leq i,j\leq 2}$ are the unique symmetric positive-definite solutions to the stochastic algebraic Riccati equation (SARE) systems:

[TABLE]

and where $(\tilde{Y}^{1},\tilde{Y}^{2})=:\tilde{Y}$ is a stochastic process defined by

[TABLE]

with $\Xi_{1},\Xi_{2}$ constants defined by

[TABLE]

and $T$ is a stochastic process in $\mathbb{R}^{2}$ defined by

[TABLE]

The associated cumulative productions $X^{*}$ and $Q^{*}$ are given in mean by

[TABLE]

where $\Gamma_{\tau}=e^{-(\Xi^{\prime}_{2}-\rho\,\text{id})\tau}$ for $\tau>0$ and $\Xi^{\prime}_{2}$ is the transpose of $\Xi_{2}$. Moreover, when the price process has a stationary level, i.e. $\lim_{t\rightarrow\infty}\mathbb{E}[P_{t}]$ $=$ $\bar{P}$ for some $\bar{P}$ $>$ [math], then the optimal cumulative productions also admit stationary levels:

[TABLE]

Interpretation and comments. We concentrate on the long-term investment decision of the social planner. First, the social planner invests in centralised generation with the same decision rule as the firm. The investment level is not necessarily the same as the firm since it depends on the level of distributed generation. But the arbitrage is guided by the same rule. The social planner satisfies the residual demand of the consumer with centralised energy. Thus, the main point is the arbitrage made by the social planner regarding distributed generation. Second, the long-term investment in distributed generation depends on the demand $D$. The long-term distributed generation capacity is a linear function of the long-term energy price $\bar{P}$, but we notice that the set of possible asymptotic Pareto optima does not cover the interval $(0,D)$ when varying $\bar{P}$. If the carbon tax is high enough compared to the distributed energy cost, there is a minimal long-term distributed capacity which will be installed even if the centralised energy price is null. Further, the Pareto optimal long-run level of distributed generation decreases linearly with distributed energy per unit cost $c$. Thus, for any given target level of distributed energy, there is a distributed energy cost under which subsidies are no longer required to achieve an asymptotic Pareto optimum. This result somehow tempers the above cited conclusion of Green and Léautier (2015) [17]. Moreover, the carbon emission limit prices $\pi_{0}$ and $\pi_{1}$ have no effect on the system (3.27) which gives the constant $K^{11}$ to $K^{22}$ because they appear divided by $D$ which is large. Thus, it is possible to analyse the effect of the absence of carbon taxes $\pi_{0}$ and $\pi_{1}$ by simply setting them to zero in the formula of the long-term investment in distributed energy and get:

[TABLE]

where $\check{K}^{11}$ is the value of $K^{11}$ of system (3.27) when $\pi_{0}=\pi_{1}=0$. Further, the investment in distributed generation by the social planner has the same expression as the investment by the consumer. But, now the social planner does not compare the price of the centralised energy with the price of the distributed energy. The social planner compares the cost of the transmission infrastructure ($\theta\bar{P}$) to the price of the distributed energy. The constant $\check{K}^{11}$ does not depend on $\theta$. Thus, the social planner invests in distributed energy only if $\theta>\frac{\rho c}{b\bar{P}}$. The cost of infrastructure should exceed the ratio of prices between distributed and centralised generations. Thus, the social planner sees the distributed energy as a substitute for transmission infrastructure. If the former condition is satisfied, the long-term investment in distributed energy will depend on the relative costs of the two energy sources because they are taken into account in the constant $K^{11}$. Unfortunately, this constant does not allow a simple expression. Nevertheless, one can note that in the case where the social planner is faced with a pure technology arbitrage problem to satisfy consumer demand, that is, there is no transmission infrastructure costs, the planner will not invest in distributed energy, whatever the relative costs of the two technologies. This unexpected result is not a consequence of the McKean-Vlasov term of variance penalisation. It holds true even if $\eta$ equals zero. This behaviour is explained by the effect both technologies have on the demand satisfaction constraint. In the long run, random distributed capacity always results in a positive term \mathbb{E}\big{[}\int_{0}^{\infty}e^{-\rho t}(D-X_{t})^{2}dt\big{]}, whereas dispatchable centralised technology can cancel it out.

Numerical illustration. With the same parameter values as in Section 3.1.2, when there is no carbon tax, a long-run price of centralised energy of $\bar{P}$ equal to $114$ €/MWh is required to justify an investment by the social planner in distributed capacity. This price has to be put into perspective with the price paid for energy by French consumers which is about 80 €/MWh or with the present electricity price in the wholesale market which is presently close to 50 €/MWh.

4 Looking for an equilibrium price

Our chief goal is to endogenise the electricity price $P$ through equilibrium considerations which result from the interactions between the players in the centralised and distributed generation markets. As seen in the previous section, the social planner can achieve a large set of long-run situations between centralised and distributed generations by choosing a regulated price of centralised energy. Our purpose is to analyse what situations can be naturally obtained by self-interested players exposed to a price signal. We address two types of equilibrium: Pareto and Stackelberg.

4.1 Asymptotic Pareto efficiency

We recall the notations from the previous sections: $\hat{X}$ $=$ $\hat{X}(P)$ is the optimal distributed generation production with control $\hat{\alpha}$ $=$ $\hat{\alpha}(P)$ for the consumer, given the price $P$ for the centralised energy of the firm, $\hat{Q}$ $=$ $\hat{Q}(\hat{X}(P))$ is the optimal centralised generation capacity with optimal control $\hat{\nu}$ $=$ $\hat{\nu}(\hat{X}(P))$ for the firm given the optimal distributed generation capacity $\hat{X}$ of the consumer, and finally $(X^{*},Q^{*})$ $=$ $(X^{*}(P),Q^{*}(P))$ is the optimal pair of distributed/centralised generation capacities with control $(\alpha^{*},\nu^{*})$ $=$ $(\alpha^{*}(P),\nu^{*}(P))$ which is determined by the social planner given the price $P$ received by the firm. Thus, a Pareto optimum is a price process $P^{Par}$ such that the optimal installed capacities for the consumer and firm coincide with the ones of the social planner, in other words, such that:

[TABLE]

Further, the second relation on the optimal centralised production is always satisfied for any price process $P$. Indeed, by definition of the social planner’s problem, given a fixed $P$, we have:

[TABLE]

which implies that $J_{2}(\nu^{*};X^{\alpha^{*}})$ $\leq$ $J_{2}(\hat{\nu}(X^{\alpha^{*}});X^{\alpha^{*}})$. Since $\hat{\nu}(X^{\alpha^{*}})$ is the unique optimal control of $\inf_{\nu\in{\cal V}}J_{2}(\nu;X^{\alpha^{*}})$, this shows that $\nu^{*}$ $=$ $\hat{\nu}(X^{\alpha^{*}})$, i.e.

[TABLE]

The first relation (4.1) on the optimal distributed generation is not satisfied in general for a given price $P$ (this is due to the fact that in general ${\rm arg}\min_{\alpha\in{\cal A}}J(\alpha,\nu)$ $\neq$ ${\rm arg}\min_{\alpha\in{\cal A}}J_{1}(\alpha)$), and the existence and determination of such a Pareto optimum price $P^{Par}$ ensuring this relation is a challenging issue that we are unfortunately not able to solve. The complex formulae for $\hat{X}(P)$ and $X^{*}(P)$ in terms of the price process $P$ seem intractable for finding a Pareto optimum satisfying the relation (4.1) at any time $t$, and instead, we propose a weaker idea of Pareto efficiency on the asymptotic (stationary) level.

Definition 4.1**.**

An asymptotic Pareto optimum price is a constant $\bar{P}^{Par}$ such that

[TABLE]

An asymptotic Pareto optimum price $\bar{P}^{Par}$ is said to be admissible if

[TABLE]

Compared to the classical Pareto condition (4.1), in (4.3) we require the weaker condition that the consumer’s optimal asymptotic average capacity coincides with the social planner’s optimal asymptotic average capacity. The admissibility conditions in (4.4) formulate on one hand the natural requirement that an asymptotic Pareto optimum price should be positive. On the other hand, we also require that the asymptotic average stationary level of consumer’s installed capacity should not exceed her demand since otherwise, it would mean that the consumer stably sells energy to the firm. The last admissibility condition states that the firm installed capacity should not be negative on the long-term.

We provide existence and characterisation results for an admissible asymptotic Pareto optimum price.

Theorem 4.1**.**

Let (3.3), (3.5) and (3.25) hold. A necessary and sufficient condition for the existence of an admissible asymptotic Pareto optimum price is that

[TABLE]

where $K$ and $K^{11}$ are the constants defined in (3.15) and (3.27). In this case, the unique admissible asymptotic Pareto optimum price is given by

[TABLE]

The corresponding equilibrium production is:

[TABLE]

Proof. In order to simplify the notations, we set

[TABLE]

We get (4.6) by solving the linear equation in (4.3) from the expressions of the stationary production level for the consumer and social planner in (3.16) and (3.29). We now verify when the admissibility conditions (4.4) are satisfied. We have $\bar{P}^{\text{Par}}>0$ if and only if

[TABLE]

Moreover, we have $\overline{X_{\infty}^{*}}(\bar{P}^{Par})$ $\in$ $(0,D)$ if and only if

[TABLE]

that is,

[TABLE]

Further, we have $\overline{Q_{\infty}^{*}}(\bar{P}^{Par})$ $>$ [math] if and only if $\pi_{1}<2\lambda D$, and we then obtain (4.1) by combining the three sets of conditions. $\Box$

Interpretation and comments. The conditions in Theorem 4.1 are verified under reasonably weak conditions for the parameters. Indeed, the first inequality in the first system in (4.1) is clearly satisfied when $\chi_{1}>0$ (notice that $\chi_{2}>0$ by (3.25) and $K^{11}>0$): hence, a sufficient condition for the existence of an asymptotic Pareto optimum price is:

[TABLE]

which is satisfied under mild and practically reasonable assumptions on the coefficients, namely $\pi_{1}>0$ and $D$ is big enough. In the asymptotic Pareto optimum, the demand level $D$ is large compared to other parameters and thus, it suggests to take the limit $D\rightarrow\infty$:

[TABLE]

where $\check{K}=\{\check{K}^{ij}\}_{1\leq i,j\leq 2}$ is a symmetric definite positive matrix solution to

[TABLE]

The price of the asymptotic Pareto optimum is positive as soon as the carbon tax $\pi_{1}$ is high enough. The higher the carbon tax the higher the Pareto price is. This finding holds true also for the distributed energy cost $\rho c/b$ because $\check{K}^{11}-K$ is positive in practical numerical cases.

Social effect of carbon tax. In the absence of a carbon tax, that is, $\pi_{0},\pi_{1}=0$, we have

[TABLE]

In particular, we see that the limiting distributed production is always negative, and thus the asymptotic Pareto optimum price is not admissible. Since a classical (in the sense of (4.1)) Pareto optimum price is obviously an asymptotic Pareto equilibrium, this means that there cannot exist admissible Pareto optimum price if $\pi_{0}=\pi_{1}=0$. In other words, the presence of a carbon tax is necessary to get admissible Pareto efficiency. The reason for this result comes from the way infrastructure costs impact differently the consumer and the social planner’s decision to invest in distributed energy. The consumer invests as soon as the total energy cost of centralised energy $(1+\theta)\bar{P}$ is higher than the cost of distributed energy $\rho c/b$ while the social planner only needs $\theta\bar{P}$ to be higher than the cost of distributed energy. It results that the consumer has a lower investment threshold price than the social planner, making agreements possible only for negative values of distributed generation. Adding carbon tax to centralised energy allows to decrease the investment threshold price of the social planner. If the carbon tax is large enough, then the social planner investment threshold price can become lower than the investment threshold of the consumer, opening the door for an equilibrium.

4.2 Asymptotic Stackelberg equilibrium

We consider here a Stackelberg equilibrium where the firm is viewed as a leader that offers a price to the consumer (the follower). The firm knows the optimal distributed generation $\hat{X}$ $=$ $\hat{X}(P)$, and searches for an equilibrium price that minimises its total cost in (2.4): $J_{2}(\nu;\hat{X}(P))$ $=$ $J_{2}(\nu,P;\hat{X}(P))$ where we now stress the dependence of the cost on the price. The Stackelberg problem for the firm is then formulated as:

[TABLE]

and if an optimal control $(\check{\nu},\check{P})$ to this problem exists, we say that $\check{P}$ is a Stackelberg equilibrium price. Unfortunately, given the expression of $\hat{X}(P)$ that can be obtained from Theorem 3.1, the problem (4.2) seems intractable, so we define an alternative and more tractable concept of Stackelberg equilibrium.

Consider a stationary situation, where the price is a constant $p$, the centralised installed capacity is a constant $q$, and the consumer’s distributed capacity is a constant $x$. Then, the cost for the firm is equal to:

[TABLE]

Because the situation is static, no installation costs appear. From Proposition 3.1 and Theorem 3.2, given a stationary price $\bar{P}$, the optimal stationary installed distributed capacity for the consumer is $\overline{\hat{X}_{\infty}}(\bar{P})$ and the optimal stationary production for the firm is $\overline{\hat{Q}_{\infty}}(\bar{P})$, where

[TABLE]

Definition 4.2**.**

An asymptotic Stackelberg equilibrium price is a constant $\bar{P}^{Sta}$ such that

[TABLE]

An asymptotic Stackelberg equilibrium price $\bar{P}^{Sta}$ is said to be admissible if

[TABLE]

Notice that, unlike the Pareto case, a Stackelberg equilibrium is not necessarily an asymptotic Stackelberg equilibrium. We provide existence and characterisation results for an admissible asymptotic Stackelberg equilibrium price.

Theorem 4.2**.**

Let (3.3) and (3.5) hold. A necessary and sufficient condition for the existence of an admissible asymptotic Stackelberg equilibrium price is that:

[TABLE]

where we recall that $K$ is the constant defined in (3.15). In this case, the equilibrium is unique and given by

[TABLE]

Proof. Observing that there is a one-to-one correspondence between $\bar{P}$ $\in$ $\mathbb{R}$ and $x$ $=$ $\overline{\hat{X}_{\infty}}(\bar{P})$ $\in$ $\mathbb{R}$, we see that

[TABLE]

where $\bar{P}(x)$ is the inverse function of $\bar{P}$ $\mapsto$ $\overline{\hat{X}_{\infty}}(\bar{P})$ given by $\bar{P}(x)$ $=$ \frac{1}{1+\theta}\big{(}-2\sigma^{2}K(D-x)+2\sigma^{2}KD+\frac{\rho c}{b}\big{)}, and $\bar{Q}(x)$ $=$ $\overline{\hat{Q}_{\infty}}(\bar{P}(x))$ $=$ $(1-\frac{\pi_{1}}{2\lambda D})(D-x)$. Hence, we see that

[TABLE]

and a minimum for $C_{F}$ exists if and only if

[TABLE]

Under such an assumption, the minimum is reached at $x^{Sta}$ as defined by

[TABLE]

which corresponds to an asymptotic Stackelberg price

[TABLE]

We now investigate the admissibility of such an asymptotic Stackelberg equilibrium price. First, under the condition $\chi$ $>$ [math], we notice that $\bar{P}^{Sta}$ $>$ [math] if and only if

[TABLE]

Secondly, we find that $\overline{\hat{X}_{\infty}}(\bar{P}^{Sta})$ $=$ $x^{Sta}$ $\in$ $(0,D)$ if and only if $D-x^{Sta}$ $\in$ $(0,D)$, i.e.

[TABLE]

that is (recall that $\chi>0$)

[TABLE]

Further, $\overline{\hat{Q}_{\infty}}(\bar{P}^{Sta})$ $=$ $\bar{Q}(x^{Sta})$ $>$ [math] if and only if $\pi_{1}<2\lambda D$. By combining all these conditions, we obtain the existence of an admissible asymptotic Stackelberg equilibrium if and only if:

[TABLE]

$\Box$

Interpretation and comments. The admissibility conditions (4.8) are verified under reasonably weak conditions for the parameters, in particular that the demand $D$ is large enough. Thus, if the demand goes to infinity, the asymptotic Stackelberg equilibrium price tends to go to infinity, contrary to the asymptotic Pareto optimum which admits a finite limit. Thus, for $D$ sufficiently large, the Stackelberg long-term price should exceed the Pareto optimum. We use a numerical example to illustrate the discrepancy between the situation where the two players try to agree in a leader/follower game and the situation where the social planner tries to find a price that will suit both interest.

Numerical illustration. Using the same parameter values as in sections 3.1.2 and 3.3 and considering that the carbon tax limits are $\pi_{0}=0$, $\pi_{1}=100$ €/MWh, the residual demand satisfaction constraints penalty is $\lambda=100\times 8760$ €/MW2/year, and a total demand of $D=50$ GW, we find an asymptotic Pareto optimum price according to formula (4.6) of 91 €/MWh and a long-term distributed capacity of 1.1 GW. On the other hand, the asymptotic Stackelberg equilibrium yields a long-term distributed capacity of 25.7 GW and a long-term price of 424 €/MWh. The Stacklelberg equilibrium results in a significant over-investment in distributed generation compared to the Pareto optimum. This over-investment can be explained by the high price charged by the firm for its centralised energy. Indeed, in the Stackelberg situation, the firm knows that whatever the consumer’s investment in distributed generation, distributed generation will not always be enough to satisfy consumer’s demand because the uncertainty of her generation is proportional to the installed capacity of distributed generation. When there is no sun, there is no generation whatever the number of installed solar panels. Thus, the firm can charge the consumer a high price each time consumer’s demand requires centralised generation. Knowing this risk, the consumer tries to reduce her residual demand by investing in a high quantity of distributed generation while taking into account her own aversion to its variance. Multiplying by 10 the variance penalty parameter results only in a small reduction of distribution generation to $25.2$ GW but to a large increase in the asymptotic Stackelberg price to 1115 €/MWh. The more the consumer is averse to variance, the more the firm can charge her a high price for its dispatchable centralised generation.

Social effect of carbon tax. We conclude this section by discussing the particular case when there is no carbon tax, that is, $\pi_{0},\pi_{1}=0$. In this case,

[TABLE]

and the admissibility conditions reads as $2\sigma^{2}KD>\rho c/b$ and $\pi_{1}<2\lambda D$. Further, we make some observations.

Provided that $D$ is big enough, we have an admissible Stackelberg equilibrium (unlike the Pareto case, where we had no admissible equilibria in the case $\pi_{0}=\pi_{1}=0$). In particular, the price is always positive.

• -

The equilibrium price is an affine function of the demand. Thus, for $D$ sufficiently large, the asymptotic Stackelberg price will be greater than the Pareto one.

• -

The equilibrium production for the consumer is always smaller than $D/2$ whatever the installation cost $c$ of distributed energy. The company has clearly an interest in limiting the amount produced by the consumer, but it is remarkable to have a simple and explicit upper bound: $D/2$.

5 Extensions

We now consider an extension of the model proposed for the consumer and the social planner. In the previous sections, the consumer’s installation costs $C_{1}(\alpha_{t})$ $=$ $c\alpha_{t}+\gamma\alpha_{t}^{2}$ only consider the current choice $\alpha_{t}$. Here, we consider a learning-by-doing effect on the distributed generation per unit cost. The cumulative installed distributed capacity, that is, $\int_{0}^{t}\alpha_{s}ds$, drives the price of solar panels down, which is typically observed for renewable energy technologies (see [21]). This suggests the following new definition for the installation costs:

[TABLE]

for some $\tilde{\mu}$ $>$ [math]. We consider a linear impact form for the learning-by-doing effect instead of the exponential form for sake of tractability. Moreover, to reduce the number of variables involved, we notice that $dX^{\alpha}_{t}=b\alpha_{t}dt+\sigma X^{\alpha}_{t}dW_{t}$ implies $\mathbb{E}[\int_{0}^{t}\alpha_{s}ds]=(\mathbb{E}[X^{\alpha}_{t}]\!-\!x_{0})/b$, so that we can rewrite $\tilde{C}_{1}(\alpha)$ as $c\alpha_{t}+\gamma\alpha_{t}^{2}-\mu\alpha_{t}\mathbb{E}[X^{\alpha}_{t}]$ (we have set $\mu=\tilde{\mu}/b$).

Remark 5.1**.**

Another interpretation of the present approach is possible. Assume that $N$ consumers are present in the market. It is reasonable to assume that the installation costs for each consumer depend on the other players’ behaviour: if the consumers are on average installing new panels, the costs are smaller. We assume this decrease is quadratic with respect to the mean of the produced electricity; in other words, the installation costs for consumer $i\in\{1,\dots,N\}$ are modelled by $\gamma(\alpha^{i}_{t})^{2}+c\alpha^{i}_{t}-\mu\alpha^{i}_{t}(\frac{1}{N}\sum_{j=1}^{N}X^{\alpha^{j}}_{t})$. As $N\to\infty$ (aggregated consumers), the expression above is approximated by $\gamma\alpha^{2}_{t}+c\alpha_{t}-\mu\alpha_{t}\mathbb{E}[X^{\alpha}_{t}]$, which is the model studied in this section. * $\Box$*

We are then led to consider the following generalisation of the consumer’s problem:

[TABLE]

This problem can be solved by the same technique as above. The results for the optimal control and production are as follows.

Theorem 5.3**.**

Let (3.3) and (3.5) hold and assume that $\mu$ is small enough to satisfy

[TABLE]

with $K$ as in Theorem 3.1, and set

[TABLE]

The optimal solar panels installation rate $\hat{\alpha}_{t}^{ext}$ and the corresponding optimal cumulative production $\mathbb{E}[\hat{X}_{t}^{ext}]$ are given by the same formulas as in Theorem 3.1, with $\Lambda$ replaced by $\tilde{\Lambda}$. In particular, if there exists $\bar{P}$ $>$ [math] s.t. $\lim_{t\rightarrow\infty}\mathbb{E}[P_{t}]$ $=$ $\bar{P}$, the optimal cumulative production of distributed generation admits a stationary level:

[TABLE]

Interpretation and comments. The learning-by-doing parameter increases the value of the distributed generation limit, which is reasonable: the installation costs are smaller, so that the limit production is bigger. However, we find that the effect of $\mu$ on the asymptotic average distributed capacity is quite small. In our context, a crude but reasonable way of determining a value of $\mu$ is to consider that if all the demand is satisfied with distributed generation, then the initial cost $c$ would be divided by a factor four. Using the same data as the numerical illustration in Section 3.1.2, this reasoning would result in a value for $\mu$ $=$ $3/4\times c/D\approx 45$ which is negligible before $2\sigma^{2}K\approx 1.1\,10^{3}$.

The social planner’s problem now reads:

[TABLE]

The optimal controls are as follows.

Theorem 5.4**.**

Let (3.3) and (3.5) hold and assume that

[TABLE]

with $K^{11}$ as in Theorem 3.3. Let $\{\tilde{\Lambda}^{ij}\}_{1\leq i,j\leq 2}$ be the unique symmetric positive-definite solution to the stochastic algebraic Riccati (SARE) system

[TABLE]

The optimal installation rates $\alpha_{t}^{*,ext},u_{t}^{*,ext}$ and the corresponding optimal cumulative production $\mathbb{E}[X_{t}^{*,ext}],\mathbb{E}[Q_{t}^{*,ext}]$ are given by the same formulae as in Theorem 3.3, with the matrix $\Lambda$ replaced by $\tilde{\Lambda}$. In particular, if there exists $\bar{P}$ $>$ [math] s.t. $\lim_{t\rightarrow\infty}\mathbb{E}[P_{t}]$ $=$ $\bar{P}$, the optimal cumulative production of distributed generation admits a stationary level:

[TABLE]

Interpretation and comments. The impact of the learning-by doing effect appears in the asymptotic average distributed capacity with the correcting term $\rho\mu D/(2b)$. With the parameter value considered for $\mu$, this term has a small impact on the asymptotic Pareto optimum distributed capacity.

The definition of asymptotic Pareto optimum prices is the same; the characterisation here reads as follows.

Theorem 5.5**.**

Let (3.3), (3.5), (5.9) and (5.13) hold. A necessary and sufficient condition for the existence of an admissible asymptotic Pareto optimum price is that

[TABLE]

where $K$ and $K^{11}$ are the constants defined in (3.15) and (3.27). In this case, the unique admissible asymptotic Pareto optimum price is given by

[TABLE]

The corresponding equilibrium production is

[TABLE]

Interpretation and comments. Notice that the asymptotic Pareto price is barely impacted by the learning-by-doing parameter $\mu$. Further, the results do not change for the asymptotic Stackelberg equilibrium, as we have only changed the installation costs, which do not play any role in the definition of the Stackelberg equilibrium.

6 Conclusions

In this paper, we study the interaction between electricity producers and consumers induced by the development of affordable distributed energy sources. Our model yet simple uses recent advances in the resolution of McKean-Vlasov optimal control problems with random coefficients. We show that in the long-term, the laissez-faire strategy leads to an over-investment in distributed energy sources compared to the socially desired outcome and to a higher price for the centralised energy. This effect can be understood as the possibility for power producers to charge at high prices the insurance they offer to the consumers for their dispatchable generation. Moreover, we show the crucial role of a significant carbon tax to justify the investment in distributed energy for the social planner. Although robust with respect to the energy price modelling, these results can be questioned regarding the reduction in the interaction between producers and consumers with distributed generation to an interaction between only two players. Indeed, competition between producers together with the possibility of energy exchanges between consumers might reduce the market power of electricity producers and help achieve a more socially and balanced equilibrium between them. These two points are left for further research.

Appendix A Appendix

A.1 Proof of Theorem 3.1

Given a candidate in the form of (3.7), our goal is to set the coefficients $K,\Lambda,Y,R$ so as to satisfy the condition in (3.9). By applying the Itô formula and completing the square with respect to the control, we get (3.1.1), where the coefficients are defined by

[TABLE]

Condition (3.11) then leads to the following equations for the coefficients:

[TABLE]

The first and second equations admit constant strictly positive solutions $K_{t}\equiv K$ and $\Lambda_{t}\equiv\Lambda$, with $K,\Lambda$ as in (3.15). The third and the fourth equations are linear Backward Stochastic Differential Equations (BSDEs), with the immediate solution (for the coefficient $Y_{t}$, first compute $Y_{t}-\mathbb{E}[Y_{t}]$) given by:

[TABLE]

Furthermore, $Y_{t}$ is well-defined by the Holder inequality and (3.5), whereas (A.2) will imply that $R_{t}<\infty$. We get (3.13) by plugging (A.1) into (3.12), while (3.14) immediately follows by (2.1) and (3.13). By the procedure above, conditions (ii) and (iii) in Lemma 3.1 are verified (we will later prove that $\hat{\alpha}$ actually lies in ${\cal A}$). As for condition (i), we have to prove that

[TABLE]

It immediately follows from the definition that $\mathbb{E}[e^{-\rho t}|R_{t}|]\to 0$ as $t\to\infty$; as for the process $Y$, we prove the stronger condition

[TABLE]

By Jensen’s inequality, Fubini’s theorem, the law of iterated conditional expectations, and (3.5), we have:

[TABLE]

We deal with the other terms in $Y_{t}$ by the same arguments and obtain the required integrability condition (A.2). Let us finally show that $\hat{\alpha}\in{\cal A}$, i.e. $\int_{0}^{\infty}e^{-\rho t}\mathbb{E}[|\hat{\alpha}_{t}|^{2}]dt$ $<$ $\infty$. Recall that $\hat{\alpha}$ is written as

[TABLE]

with

[TABLE]

By Itô’s formula to (2.1) for $\alpha$ $=$ $\hat{\alpha}$, Young’s inequality, and Gronwall’s lemma, we have:

[TABLE]

for all $\epsilon$ $>$ [math]. For $\epsilon$ small enough, and under (3.3), we have $\rho-\sigma^{2}-b\epsilon$ $>$ [math], and thus it suffices to show that $\int_{0}^{\infty}e^{-\rho t}\mathbb{E}[|U_{t}|^{2}]dt$ $<$ $\infty$ to ensure the square-integrability condition: $\int_{0}^{\infty}e^{-\rho t}\mathbb{E}[|\hat{X}_{t}|^{2}]dt$ $<$ $\infty$, and so $\hat{\alpha}$ $\in$ ${\cal A}$. From the integrability condition (A.2) on $Y$, we then have to check that \int_{0}^{\infty}e^{-\rho t}\big{(}\mathbb{E}[\hat{X}_{t}]\big{)}^{2}dt $<$ $\infty$, hence by (3.14) that (we set $B$ $=$ $b^{2}\Lambda/\gamma$)

[TABLE]

By Fubini-Tonelli we can rewrite $I$ as:

[TABLE]

by writing $e^{-(\rho+B)u}$ $=$ $e^{-2\epsilon u}e^{-(\rho+B-2\epsilon)u}$ for $\epsilon>0$, by Jensen’s inequality w.r.t. the measure $e^{-2\epsilon u}du$ and by Fubini-Tonelli, we get:

[TABLE]

which finally leads to

[TABLE]

Hence, from (3.5) and for $\epsilon$ small enough we have $I$ $<$ $\infty$ and then $\hat{\alpha}$ $\in$ ${\cal A}$. $\Box$

A.2 Proof of Theorem 3.2

A suitable adaptation of Lemma 3.1 holds. We look for a candidate in the form of:

[TABLE]

where the dynamics of the coefficients $K,Y,R$ are given by

[TABLE]

for some deterministic process $\dot{K}$ and $\mathbb{F}^{0}$-adapted processes $\dot{Y}$, $\dot{R}$, $Z^{Y}$, $Z^{R}$. As in Appendix A.1, we have assumed the quadratic coefficient to be deterministic. Moreover, since the randomness comes from the $\mathbb{F}$-adapted process $X^{\alpha}$, the stochastic part in $Y,R$ only depends on $W$.

By applying Itô’s formula to $S_{t}^{\nu}$ $=$ $e^{-\rho t}v_{t}(Q_{t}^{\nu})$ $+$ $\int_{0}^{t}e^{-\rho s}\ell_{s}(Q^{\nu}_{s},\nu_{s})ds$ and completing the square, we get $d\mathbb{E}[S_{t}^{\nu}]=e^{-\rho t}\mathbb{E}[{\cal D}_{t}^{\nu}]dt$, with

[TABLE]

where the coefficients are defined by

[TABLE]

Setting $\mathbb{E}[\mathcal{D}^{\nu}_{t}]\geq 0$ for each $\nu$ and $\mathbb{E}[\mathcal{D}^{\nu}_{t}]=0$ for $\nu=\hat{\nu}$ provides the optimal control

[TABLE]

and the following equations for the coefficients:

[TABLE]

The first equation admits a constant solution $K_{t}\equiv\tilde{K}$, with $\tilde{K}$ as in (3.21), while the second and third equations are linear BSDEs, with immediate solutions:

[TABLE]

We get (3.19) by (A.4) and (A.2), whereas (3.20) immediately follows by (3.19) and (2.3). Moreover, the conditions (i) and (iii) in Lemma 3.1 are verified by standard estimates as in Appendix A.1. Finally, we get the limit result by the same computations as the ones in Proposition 3.1. $\Box$

A.3 Proof of Theorem 3.3

A suitable adaptation of Lemma 3.1 holds. We use vector notations and a small change of variable by setting:

[TABLE]

so that the dynamics of $Z^{\delta}$ are written as

[TABLE]

The payoff is rewritten as $\tilde{g}_{t}(Z^{\delta}_{t},\mathbb{E}[Z^{\delta}_{t}],\delta_{t})$, where

[TABLE]

where ′ denotes transposition and the coefficients are defined by (notice that $Q,\tilde{Q},N,U$ are constant, $T$ is stochastic and $\mathbb{F}^{0}$-adapted)

[TABLE]

Correspondingly, we consider candidates in the form of:

[TABLE]

where the dynamics of the coefficients $K,\Lambda,Y,R$ are given by

[TABLE]

for some symmetric deterministic processes $\dot{K},\dot{\Lambda}$ and $\mathbb{F}^{0}$-adapted processes $\dot{Y}$, $\dot{R}$, $Z^{Y}$, $Z^{R}$. As in Appendix A.1, we have assumed the quadratic coefficient to be deterministic. Moreover, since the randomness only comes from the $\mathbb{F}^{0}$-adapted process $T$, the stochastic part in $Y,R$ only depends on $W^{0}$.

By applying Itô’s formula to $S_{t}^{\delta}$ $=$ $e^{-\rho t}v_{t}(Z_{t}^{\delta},\mathbb{E}[Z_{t}^{\delta}])$ $+$ $\int_{0}^{t}e^{-\rho s}\tilde{g}_{s}(Z^{\delta}_{s},\mathbb{E}[Z^{\delta}_{s}],\delta_{s})ds$ and completing the square, we get $d\mathbb{E}[S_{t}^{\delta}]=e^{-\rho t}\mathbb{E}[{\cal D}_{t}^{\delta}]dt$, with

[TABLE]

where the coefficients are defined by

[TABLE]

Setting $\mathbb{E}[\mathcal{D}^{\delta}_{t}]\geq 0$ for each $\delta$ and $\mathbb{E}[\mathcal{D}^{\delta}_{t}]=0$ for $\delta=\delta^{*}$ provides the optimal control

[TABLE]

and the following equations for the coefficients

[TABLE]

The first and the second equation admit constant solutions $K_{t}\equiv K$ and $\Lambda_{t}\equiv\Lambda$: there exists a unique couple of symmetric positive-definite matrices $(K,\Lambda)$ which solves the equations

[TABLE]

i.e. the systems (3.27) (we postpone the proof to the end of this appendix). The third and the fourth equations are linear BSDEs, with immediate solution (for the coefficient $Y_{t}$, first compute $Y_{t}-\mathbb{E}[Y_{t}]$):

[TABLE]

Since $K,\Lambda>0$, the matrices $\rho\,\text{Id}+KN^{-1}$ and $\rho\,\text{Id}+\Lambda N^{-1}$ are positive-definite and invertible; then, by arguing as in Appendix A.1, the processes $Y_{t}$ and $R_{t}$ are well-defined. We get (3.3) by plugging (A.3) into (A.9) and setting $\tilde{Y}=Y+U$, whereas (3.28) immediately follows by (A.5) and (3.3). Moreover, the conditions (i) and (iii) in Lemma 3.1 are verified by standard estimates as in Appendix A.1. Further, we get the limit result by the same computations as the ones in Proposition 3.1. $\Box$

Lemma A.1**.**

There exists a unique couple of symmetric positive-definite matrices $(K,\Lambda)$ which solves the equations (A.10)-(A.11).

Proof.

Uniqueness immediately follows from (A.6). To prove the existence of the solutions, we link (A.10)-(A.11) to suitable control problems. For $T\in\mathbb{R}^{+}\cup\{\infty\}$ and $x\in\mathbb{R}^{2}$, we consider

[TABLE]

By arguing as in (3.4), it is easy to see that $u\in\mathcal{U}_{T}$ implies \mathbb{E}\big{[}\int_{0}^{T}e^{-\rho s}|X_{s}|^{2}ds\big{]} $<$ $\infty$, so that the problems are well-defined. If $T$ is finite, we know (see [23, Sections 6.6 and 6.7], with a straightforward adaptation of the arguments to include the discount factor) that there exists a unique solution $\{K_{t;T}\}_{t\in[0,T]}$ to

[TABLE]

and that for every $x\in\mathbb{R}^{2}$ we have

[TABLE]

It is easy to see that $V_{T}\to V_{\infty}$ as $T\to\infty$; as a consequence, there exists

[TABLE]

By a classical argument and since the functions $\{K_{t;T}\}_{t\in[0,T]}$ solve (A.13), $\tilde{K}$ is a solution to (A.10). Also, $\tilde{K}$ is symmetric as it is the limit of symmetric matrices. Moreover, notice that $N^{-1}>0$ and assume that $Q>0$; by standard arguments in control theory, it follows that $x^{\prime}\tilde{K}x=V_{\infty}(x)>0$ for each $x\neq 0$, so that $\tilde{K}>0$. Hence, we have proved that (A.10) admits a symmetric positive-definite solution provided that $Q>0$, which is true by (3.25).

By similar arguments, (A.11) admits a symmetric positive-definite solution provided that $SKS+Q+\tilde{Q}>0$, which is true by (3.25), since $K^{11}>0$. ∎