Pricing in non-convex markets with quadratic deliverability costs

TL;DR

This paper extends market-clearing price methods to include quadratic costs, such as ramping costs in renewable energy, providing a more accurate pricing framework for non-convex electricity markets.

Contribution

It introduces a novel approach to incorporate quadratic costs into market-clearing prices in non-convex markets, especially electricity markets with renewable ramping costs.

Findings

Derived new pricing formulas accounting for quadratic costs.

Analyzed the impact of ramping costs on market prices.

Validated results with a classical market example.

Abstract

The problem of obtaining market-clearing prices for markets with non-convexities has been widely studied in the literature. This is particularly the case in electricity markets, where worldwide deregulation leads to markets in which non-convexities arise from the decisions of market operators regarding which generators are committed to provide electricity power. Here, we extend seminal results in this area to address the problem of obtaining market-clearing prices for markets in which beyond non-convexities, it is relevant to account for convex quadratic market costs. In a general market, such costs arise from quadratic commodity costs or transactions costs. In an electricity market, such quadratic costs arise when ramping costs need to be considered due to the presence of renewable energy sources, which continue to increase their participation in electricity markets. To illustrate our results, we compute and analyze the clearing prices of a classical market problem with the addition of ramping costs.

Tables

Table 1. Table 3.1 : Characteristics of Smokestack and High Tech plants (O’Neill et al.,, 2005 ) .

	(type 1 plant)	(type 2 plant)
Characteristics	Smokestack	High Tech
Capacity	16.00	7.00
Construction cost	53.00	30.00
Marginal cost	3.00	2.00
Average cost at capacity	6.31	6.28
Total cost at capacity	101.00	44.00

Table 2. Table 3.2 : Optimal solution of Scarf’s market problem ( 19 )  (O’Neill et al.,, 2005 ) .

	Units of type			Unit’s output
Demand	1	2		1	2	Total Cost
56	0	8		0	56	352
58	1	6		16	42	365
60	2	4		32	28	378
62	3	2		48	14	391
64	4	0		64	0	404
66	2	5		31	35	419
68	3	3		47	21	432
70	0	10		0	70	440

Table 3. Table 3.3 : Market-clearing price of Scarf’s problem.

Commodity Price	Plant 1 Start-up Price	Plant 2 Start-up Price
3	53	23

Table 4. Table 3.4 : Optimal solution of modified Scarf’s problem with r 1 = 0.1 , r 2 = 0.1 formulae-sequence subscript 𝑟 1 0.1 subscript 𝑟 2 0.1 r_{1}=0.1,r_{2}=0.1 .

	Type 1 Output				Type 2 Output
	Partial		Full		Partial		Full		Cost
Demand	No.	Prod.	No.	Prod.	No.	Prod.	No.	Prod.	Ramp	Total
56	3	45.00	0	0	1	4.00	1	7	1.90	377.90
58	3	46.50	0	0	1	4.50	1	7	2.10	383.60
60	0	0	3	48	1	5.00	1	7	2.50	389.50
62	0	0	3	48	1	7.00	1	7	4.90	395.90
64	3	47.40	0	0	2	9.60	1	7	4.62	429.02
66	0	0	3	48	2	11.00	1	7	6.05	435.05
68	0	0	3	48	2	13.00	1	7	8.45	441.45
70	1	15.00	3	48	0	0	1	7	22.50	467.50

Table 5. Table 3.5 : Market-clearing price of modified Scarf’s problem with r 1 = 0.1 , r 2 = 0.1 formulae-sequence subscript 𝑟 1 0.1 subscript 𝑟 2 0.1 r_{1}=0.1,r_{2}=0.1 .

		Plant 1 Start-up Price		Plant 2 Start-up Price
Demand	Unit Price	Partial	Full & Closed	Partial	Full & Closed
56	2.80	53.00	53.00	30	24.40
58	2.90	53.00	53.00	30	23.70
60	3.00	53.00	53.00	30	23.00
62	3.40	46.60	46.60	30	20.20
64	2.96	53.00	53.00	30	23.28
66	3.10	51.40	51.40	30	22.30
68	3.30	48.20	48.20	30	20.90
70	6.00	53.00	5.00	2	2.00

Table 6. Table 3.6 : Optimal solution of modified Scarf’s problem with r 1 = 0.1 , r 2 = 0.3 formulae-sequence subscript 𝑟 1 0.1 subscript 𝑟 2 0.3 r_{1}=0.1,r_{2}=0.3 .

	Type 1 Output				Type 2 Output
	Partial		Full		Partial		Full		Cost
Demand	No.	Prod.	No.	Prod.	No.	Prod.	No.	Prod.	Ramping	Total
56	3	47.4	0	0	1	1.6	1	7	0.78	379.18
58	0	0	3	48	1	3.0	1	7	2.70	385.70
60	0	0	3	48	1	5.0	1	7	7.50	394.50
62	0	0	3	48	0	0	2	14	14.70	405.70
64	1	9.0	3	48	0	0	1	7	4.62	435.10
66	1	11.0	3	48	0	0	1	7	12.10	445.10
68	1	13.0	3	48	0	0	1	7	16.90	455.90
70	1	15.0	3	48	0	0	1	7	22.50	467.50

Table 7. Table 3.7 : Market-clearing price of modified Scarf’s problem with r 1 = 0.1 , r 2 = 0.3 formulae-sequence subscript 𝑟 1 0.1 subscript 𝑟 2 0.3 r_{1}=0.1,r_{2}=0.3 .

		Start-up Compensation
		Type 1 Output		Type 2 Output
Demand	Unit Price	Partial	Full & Closed	Partial	Full & Closed
56	2.96	53.00	53.00	30.00	23.28
58	3.80	40.20	40.20	30.00	17.40
60	5.00	21.00	21.00	30.00	9.00
62	6.20	1.80	1.80	30.00	0.60
64	4.80	53.00	24.20	10.40	10.40
66	5.20	53.00	17.80	7.60	7.60
68	5.60	53.00	11.40	4.80	4.80
70	6.00	53.00	5.00	2.00	2.00

Table 8. Table 3.8 : Optimal solution of modified Scarf’s problem with r 1 = r 2 = 1 subscript 𝑟 1 subscript 𝑟 2 1 r_{1}=r_{2}=1 .

	Type 1 Output				Type 2 Output
	Partial		Full		Partial		Full		Cost
Demand	No.	Prod.	No.	Prod.	No.	Prod.	No.	Prod.	Ramp	Total
56	0	0	3	48	1	1	1	7	1.0	380.0
58	0	0	3	48	1	3	1	7	9.0	392.0
60	0	0	3	48	1	5	1	7	25.0	412.0
62	0	0	3	48	1	7	1	7	49.0	440.0
64	0	0	3	48	2	9	1	7	40.5	465.5
66	0	0	3	48	2	11	1	7	60.5	489.5
68	0	0	3	48	2	13	1	7	84.5	517.5
70	0	0	3	48	3	15	1	7	75.0	542.0

Table 9. Table 3.9 : Market-clearing price of modified Scarf’s problem with r 1 = r 2 = 1 subscript 𝑟 1 subscript 𝑟 2 1 r_{1}=r_{2}=1 .

		Plant 1 Start-up Price		Plant 2 Start-up Price
Demand	Unit Price	Partial	Full & Closed	Partial	Full & Closed
56	4	37	37	30	16
58	8	-27	-27	30	-12
60	12	-91	-91	30	-40
62	16	-155	-155	30	-68
64	11	-75	-75	30	-33
66	13	-107	-107	30	-47
68	15	-139	-139	30	-61
70	12	-91	-91	30	-40

Equations

$$\begin{array}[]{llllllll}p_{\operatorname{MIQP}}^{\ast}=&\min&\displaystyle\sum_{k=1}^{n}c_{k}x_{k}+d_{k}z_{k}+r_{k}(x_{k}-x_{k}^{0})^{2}\\ &\operatorname{s.\!t.}&\displaystyle\sum_{k=1}^{n}a_{k}x_{k}=b_{0},\\ &&g_{k}x_{k}+h_{k}z_{k}\geq b_{k},&k=1,\dots,n,\\ &&z_{k}\in\{0,1\},&k=1,\dots,n,\\ &&x_{k}\geq 0,&k=1,\dots,n,\end{array}$$

$$\begin{array}[]{lllllllll}p_{\operatorname{MIQP}}^{\ast}=&\min&\displaystyle\sum_{k=1}^{n}c_{k}x_{k}+d_{k}z_{k}+r_{k}y_{k}\\ &\operatorname{s.\!t.}&\displaystyle\sum_{k=1}^{n}a_{k}x_{k}=b_{0},\\ &&g_{k}x_{k}+h_{k}z_{k}\geq b_{k},&k=1,\dots,n,\\ &&y_{k}\geq(x_{k}-x_{k}^{0})^{2},&k=1,\dots,n,\\ &&z_{k}\in\{0,1\},&k=1,\dots,n,\\ &&x_{k},y_{k}\geq 0,&k=1,\dots,n.\end{array}$$

$${\mathcal{K}}^{n+1}=\{(w_0,w)\in {\mathbb{R}}^{n+1}:w_0\ge \Vert (w_1,w_2,\dots ,w_n){\Vert }_2\},$$

$$y_k\ge (x_k-x_k^0{)}^2\iff \left(y_k+1,y_k-1,2(x_k-x_k^0)\right)\in {\mathcal{K}}^3.$$

$$\begin{array}[]{lllllll}p_{\operatorname{MISOCP}}^{\ast}=&\min&\displaystyle\sum_{k=1}^{n}c_{k}x_{k}+d_{k}z_{k}+r_{k}y_{k}\\ &\operatorname{s.\!t.}&\displaystyle\sum_{k=1}^{n}a_{k}x_{k}=b_{0},\\ &&g_{k}x_{k}+h_{k}z_{k}\geq b_{k},&k=1,\dots,n,\\ &&\left(y_{k}+1,y_{k}-1,2(x_{k}-x^{0}_{k})\right)\in\mathcal{K}^{3},&k=1,\dots,n,\\ &&z_{k}\in\{0,1\},&k=1,\dots,n,\\ &&x_{k},y_{k},z_{k}\geq 0,&k=1,\dots,n.\par\end{array}$$

$$z^{\ast }={\mathrm{argmin}}_{z\in \{0,1{\}}^n}\{\text{eqref}eq.p3\};$$

$$\begin{array}[]{lllllll}p_{\operatorname{SOCP}}^{\ast}=&\min&\displaystyle\sum_{k=1}^{n}c_{k}x_{k}+d_{k}z_{k}+r_{k}y_{k}\\ &\operatorname{s.\!t.}&\displaystyle\sum_{k=1}^{n}a_{k}x_{k}=b_{0},&(p_{0})\\ &&g_{k}x_{k}+h_{k}z_{k}\geq b_{k},&(q_{k})&k=1,\dots,n,\\ &&\left(y_{k}+1,y_{k}-1,2(x_{k}-x^{0}_{k})\right)\in\mathcal{K}^{3},&(\gamma_{k},\alpha_{k},\beta_{k})&k=1,\dots,n,\\ &&z_{k}=z_{k}^{*},&(p_{k})&k=1,\dots,n,\\ &&x_{k},y_{k}\geq 0,&&k=1,\dots,n.\end{array}$$

$$L(x,y,z,p_0,p,q,\gamma ,\alpha ,\beta )=\sum _{k=1}^n{L}_k(x_k,y_k,z_k,p_0,p_k,q_k,{\gamma }_k,{\alpha }_k,{\beta }_k),$$

$$\begin{array}[]{lcl}L_{k}(x_{k},\cdots,\beta_{k})&=&c_{k}x_{k}+d_{k}z_{k}+r_{k}y_{k}+p_{0}(\frac{1}{k}b_{0}-a_{k}x_{k})-q_{k}(g_{k}x_{k}+h_{k}z_{k}-b_{k})\\ &&+p_{k}(z_{k}^{*}-z_{k})-(y_{k}+1,y_{k}-1,2(x_{k}-x_{k}^{0}))^{\intercal}(\gamma_{k},\alpha_{k},\beta_{k})\\[8.61108pt] &=&(c_{k}-a_{k}p_{0}-g_{k}q_{k}-2\beta_{k})x_{k}+(d_{k}-h_{k}q_{k}-p_{k})z_{k}\\ &&+(r_{k}-\gamma_{k}-\alpha_{k})y_{k}+\frac{1}{k}b_{0}p_{0}+b_{k}q_{k}+z_{k}^{*}p_{k}-\gamma_{k}+\alpha_{k}+2\beta_{k}x_{k}^{0},\end{array}$$

$$\max_{\scriptsize\begin{array}[]{l}p_{0}\in\mathbb{R},p\in\mathbb{R}^{n},q\in\mathbb{R}^{n}_{+},\\ (\gamma_{k},\alpha_{k},\beta_{k})\in\mathcal{K}^{3},k=1,\dots,n\end{array}}\ \min_{x\geq 0,y\geq 0,z\geq 0}L(x,y,z,p,q,u,v,w,\gamma,\alpha,\beta),$$

$$\begin{array}[]{lllllll}d_{\operatorname{SOCP}}^{\ast}=&\max&b_{0}p_{0}+\displaystyle\sum_{k=1}^{n}\left(b_{k}q_{k}+z_{k}^{*}p_{k}-\gamma_{k}+\alpha_{k}+2\beta_{k}x_{k}^{0}\right)\\ &\operatorname{s.\!t.}&c_{k}-a_{k}p_{0}-g_{k}q_{k}-2\beta_{k}\geq 0,&k=1,\dots,n,\\ &&d_{k}-h_{k}q_{k}-p_{k}\geq 0,&k=1,\dots,n,\\ &&r_{k}-\gamma_{k}-\alpha_{k}\geq 0,&k=1,\dots,n,\\ &&(\gamma_{k},\alpha_{k},\beta_{k})\in\mathcal{K}^{3},&k=1,\dots,n,\\ &&q_{k}\geq 0,&k=1,\dots,n.\\ \end{array}$$

$$p_{\mathrm{MIQP}}^{\ast }=p_{\mathrm{MISOCP}}^{\ast }=p_{\mathrm{SOCP}}^{\ast }=d_{\mathrm{SOCP}}^{\ast }.$$

$$y_k+1>\Vert (y_k-1,2(x_k^{\ast }-x_k^0)){\Vert }_2,$$

$$\begin{array}[]{ll}p_{0}=\min_{k=1,\dots,n}\left\{\frac{c_{k}-g_{k}}{a_{k}}\right\},\\[8.61108pt] (p_{k},q_{k},\gamma_{k},\alpha_{k},\beta_{k})=\left(d_{k}-h_{k}-1,1,\frac{1}{2}r_{k},\frac{1}{4}r_{k},-\frac{1}{4}r_{k}\right),\end{array}$$

$${\gamma }_k=\frac{1}{2}{r}_k>\frac{1}{2\sqrt{2}}{r}_k=\Vert ({\alpha }_k,{\beta }_k){\Vert }_2,$$

$$\min$$

$$s.t.$$

$$x_k\ge 0,$$

$$z_k\in \{0,1\};$$

$$p_{{\mathrm{MISOCP}}_k}^{\ast }(t_0,t_k)=$$

$$s.t.$$

$$(y_k+1,y_k-1,2(x_k-x_k^0))\in {\mathcal{K}}^3,$$

$$x_k,y_k\ge 0,$$

$$z_k\in \{0,1\}.$$

$$0\le (c_k-a_k{p}_0^{\ast }-g_k{q}_k^{\ast }-2{\beta }_k^{\ast })\perp x_k^{\ast }\ge 0,$$

$$0\le (d_k-h_k{q}_k^{\ast }-p_k^{\ast })\perp z_k^{\ast }\ge 0,$$

$$0\le (r_k-{\gamma }_k^{\ast }-{\alpha }_k^{\ast })\perp y_k^{\ast }\ge 0,$$

$$0=p_0^{\ast }(b_0-\sum _{k=1}^n{a}_k{x}_k^{\ast }),$$

$$0\le q_k^{\ast }\perp (g_k{x}_k^{\ast }+h_k{z}_k^{\ast }-b_k)\ge 0,$$

$$0=p_k^{\ast }(z_k^{\prime }-z_k^{\ast }),$$

$$(y_k^{\ast }+1,y_k^{\ast }-1,2(x_k^{\ast }-x_k^0))\perp ({\gamma }_k^{\ast },{\alpha }_k^{\ast },{\beta }_k^{\ast }),$$

$$p_{{\mathrm{MISOCP}}_k}^{\ast }(p_0^{\ast },p_k^{\ast })=$$

$$s.t.$$

$$(y_k+1,y_k-1,2(x_k-x_k^0))\in {\mathcal{K}}^3,$$

$$x_k,y_k\ge 0,$$

$$z_k\in \{0,1\}.$$

$$\begin{array}{rl}{\hat{p}}_{{\mathrm{MISOCP}}_k}(p_0^{\ast },p_k^{\ast })=& c_k{x}_k^{\ast }+d_k{z}_k^{\ast }+r_k{y}_k^{\ast }-p_0^{\ast }{a}_k{x}_k^{\ast }-p_k^{\ast }{z}_k^{\ast }.\end{array}$$

$$\begin{array}{rl}{\hat{p}}_{{\mathrm{MISOCP}}_k}(p_0^{\ast },p_k^{\ast })=& c_k{x}_k^{\ast }+d_k{z}_k^{\ast }+r_k{y}_k^{\ast }-p_0^{\ast }{a}_k{x}_k^{\ast }-p_k^{\ast }{z}_k^{\ast }-q_k^{\ast }(g_k{x}_k^{\ast }+h_k{z}_k^{\ast }-b_k)\\ & -(r_k-{\gamma }_k^{\ast }-{\alpha }_k^{\ast })y_k^{\ast }-(y_k^{\ast }+1,y_k^{\ast }-1,2(x_k^{\ast }-x_k^0){)}^{⊺}({\gamma }_k^{\ast },{\alpha }_k^{\ast },{\beta }_k^{\ast }),\\ =& (c_k-a_k{p}_0^{\ast }-g_k{q}_k^{\ast }-2{\beta }_k^{\ast })x_k^{\ast }+(d_k-p_k^{\ast }-h_k{q}_k^{\ast })z_k^{\ast }+y_k^{\ast }({\gamma }_k^{\ast }+{\alpha }_k^{\ast })\\ & -(y_k^{\ast }{\gamma }_k^{\ast }+{\gamma }_k^{\ast }+y_k^{\ast }{\alpha }_k^{\ast }-{\alpha }_k^{\ast }-2{\beta }_k^{\ast }{x}_k^0),\\ =& q_k^{\ast }{b}_k-{\gamma }_k^{\ast }+{\alpha }_k^{\ast }+2{\beta }_k^{\ast }{x}_k^0.\end{array}$$

$$\begin{array}{rl}& q_k^{\ast }(g_k{x}_k+h_k{z}_k-b_k)\ge 0\\ & y_k(r_k-{\gamma }_k^{\ast }-{\alpha }_k^{\ast })\ge 0,\\ & (y_k+1,y_k-1,2(x_k-x_k^0){)}^{⊺}({\gamma }_k^{\ast },{\alpha }_k^{\ast },{\beta }_k^{\ast })\ge 0,\end{array}$$

Full text

**Keywords ** market-clearing prices, quadratic costs, ramping costs, unit commitment, mixed-integer quadratic programming, renewable energy sources.

1 Introduction

The presence of non-convexities is inherent in markets due to economies of scale, start-up and/or shut-down costs, avoidable costs, indivisibilities, and minimum supply requirements. These non-convexities make the problem of finding appropriate prices that result in a market equilibrium challenging. This issue has been addressed in classical work by Starr, (1969); Gomory and Baumol, (1960); Scarf, (1994, 1990). Continued work in this area has been recently reviewed by Liberopoulos and Andrianesis, (2016) and Van Vyve et al., (2011). Here, we focus on considering the effect of not only non-convexities, but also potential convex quadratic costs that affect market prices. As discussed below, these market characteristics are particularly important in deregulated electricity markets with high penetration of renewable energy sources.

In electricity deregulated markets (see, e.g., Ruiz et al.,, 2012; O’Neill et al.,, 2005), a market operator or individual system operator (ISO) receives information about generation constraints, marginal generation costs, and fixed commitment costs, from generators participating in the market. Based on this information, the ISO decides the generators that should be committed in the market, as well as their appropriate compensation.

Due to the electricity market non-convexities related to start-up/shut-down costs and minimum output requirements (among others), it is difficult for ISOs to obtain the appropriate compensation values for committed generators. One way in which ISOs can obtain an estimate of the correct compensation values is by finding the shadow (dual) prices associated to the linear programming relaxation (cf., Conforti et al.,, 2014) of the mixed-integer linear program (MILP) (cf., Conforti et al.,, 2014) associated to the market’s unit commitment problem (UC) (cf., Hobbs et al.,, 2006). The generators’ compensation values are then computed by adding uplift payments (cf., Jayantilal et al.,, 2001). However, these uplift payments may be significant enough to modify the suppliers’ incentives (Liberopoulos and Andrianesis,, 2016). To address this issue, a number of alternative pricing schemes have been recently developed by Garcıa et al., (2006); O’Neill et al., (2005); Bjørndal and Jörnsten, (2008); Hogan and Ring, (2003); Ruiz et al., (2012); Liberopoulos and Andrianesis, (2016), among others (see, Liberopoulos and Andrianesis,, 2016, for a recent review). However, to the best of our knowledge, none of these approaches directly consider potential ramping costs in the electricity market.

The penetration of renewable energy sources such as wind, solar, and wave energy into the electricity market has been steadily increasing. The U.S. Energy Information Administration estimates that in the U.S., the percentage of energy generated from renewable energy sources has increased from 9.5% in 2006 to 13.3% in 2015. As a result, ISO’s commitment decisions require conventional generators to ramp up or down regularly due to the variability in the power that renewable sources can provide throughout a given day (see, e.g., Sioshansi and Short,, 2009). At the same time, ramping results in substantial wear and tear costs (see, e.g., Lew et al.,, 2012). Therefore, considering these ramping costs in the computation of compensation values is becoming necessary to provide adequate settlement prices for the different generators in the electricity market.

In order to take into account ramping costs in the electricity market, we extend the seminal results introduced by O’Neill et al., (2005) to obtain appropriate prices in markets with non-convexities. O’Neill et al., (2005) show that after solving the MILP associated with the ISO’s UC; the desired clearing prices can be obtained from the shadow prices of the demand and capacity constraints of the linear program resulting from fixing the commitment (binary) variables of the ISO’s UC to their optimal value.

Here, we extend this approach by considering ramping costs (or in more generality, quadratic convex costs) into the formulation of the unit commitment problem. Then, using convex optimization techniques (cf., Boyd and Vandenberghe,, 2004), we obtain the market-clearing prices in the presence of ramping costs from the dual variables values (shadow prices) associated with the optimal solution of an appropriate convex optimization problem. That is, one obtains a set of prices that satisfy the market equilibrium conditions in which suppliers would not want to change their energy dispatch at these prices. With these results in hand, we perform numerical experiments to show the impact of ramping costs on the clearing prices of a market with non-convexities.

To make the article more self-contained, we introduce some of the basic convex optimization tools used in the article and provide appropriate references for further relevant results in this area.

The pricing results are mainly motivated by an ISO looking to obtain clearing prices in an electricity market with renewable energy sources. However, following O’Neill et al., (2005), we use the general formulation of a bidding market in which the auctioneer is buying and/or selling a commodity, and has an objective of maximizing the value to bidders, when potential quadratic commodity costs (e.g., in a labor market (Bentolila and Saint-Paul,, 1992)) or quadratic transaction costs (e.g., in a finance market (Olivares-Nadal and DeMiguel,, 2015; Brown and Smith,, 2011) are present in the market).

It is worth to mention that related advances on obtaining market-clearing prices in markets with non-convexities have been done recently. For example, consider the work of: Zoltowska, (2016), who considers demand shifting bids and transmission constraints in the market; Ye et al., (2015), who consider, from the point of view of the customer, non-convexities arising when considering flexible demand options (e.g., the option to forgo demand); Van Vyve et al., (2011), who propose a new model to obtain prices in a market with non-convexities by combining the approaches used in both the US and European electricity markets; and Muatore, (2008), who proposes an algorithm to find market-clearing prices in markets with non-convexities that provide incentives to both maintain and increase generating capacity.

The rest of the article is organized as follows. In Section 2, we introduce the market’s assignment problem in the presence of convex quadratic costs. Furthermore, we generalize the results of O’Neill et al., (2005) to obtain clearing prices for the market. In Section 3, we illustrate our results by computing and analyzing the clearing prices associated to Scarf’s classical market problem (Scarf,, 1994) when potential ramping costs are taken into account. In Section 4, we finish with some concluding remarks.

2 Market problem with convex quadratic costs

Consider a bidding market, where the auctioneer is buying and/or selling a commodity, and has an objective of maximizing the value to bidders, when potential transaction convex quadratic costs are present in the market. Following O’Neill et al., (2005), the market’s assignment problem can be formulated as the following mixed-integer quadratic program (MIQP) (cf., Lazimy,, 1982):

[TABLE]

where for any market bidder $k=1,\dots,n$: $x_{k}\in\mathbb{R}_{+}$ represents the units of commodity provided by the bidder; $z_{k}\in\{0,1\}$ indicates whether the bidder is committed to provide units of commodity; $c_{k},d_{k}\in\mathbb{R}$, are respectively the variable and fixed costs associated with the bidder’s activities; $a_{k}\in\mathbb{R}$ reflects the production or demand characteristics of the bidder in the market-clearing constraint $\sum_{k=1}^{n}a_{k}x_{k}=b_{0}$, where $b_{0}\in\mathbb{R}$ is the amount of commodity to be auctioned, with $b_{0}\neq 0$ in a one-sided auction and $b_{0}=0$ in a two-sided auction; $g_{k},h_{k}\in\mathbb{R}$ reflect restrictions on the bidder’s operations (e.g., production of a particular plant is limited to the capacity of that plant); $b_{k}\in\mathbb{R}$ represents the right hand sides of the internal constraints of the bidder. Also, in an extension of the market assignment problem in O’Neill et al., (2005, Sec. 5), $r_{k}\in\mathbb{R}_{+}$ denotes the quadratic costs associated with the deviation of the commodity provided by bidder $k=1,\dots,n$ from a target or previous commodity production level $x_{k}^{0}\in\mathbb{R}$. The parameters $r_{k}\in\mathbb{R}_{+},x_{k}^{0}\in\mathbb{R}$ can also be used to model quadratic commodity costs, as well as quadratic transaction costs.

Notice that by assuming that $r_{k}\geq 0$ for $k=1,\dots,n$, one ensures that (1) has a convex quadratic objective. Also, note that if $r_{k}=0$ for $k=1,\dots,n$, problem (1) is equivalent to the PIP problem in O’Neill et al., (2005, Sec. 5) when bidders are assumed to provide a single commodity. This single-commodity assumption is made here for ease of presentation and to identify the commodity with power in electricity markets. However, all the results presented thereof extend in straightforward fashion to the more general multi-commodity market considered in O’Neill et al., (2005).

In what follows, we assume that problem (1) is feasible; that is, there is an assignment of the bidders that satisfies both the operating constraints of the bidders as well as the market-clearing constraint. Also, without loss of generality, we assume that $a_{k}\neq 0$, $k=1,\dots,n$ (i.e., bidders that do not contribute to the market-clearing are not considered), and $r_{k}>0$, $k=1,\dots,n$ (i.e., bidders that do not incur ramping costs do not need a corresponding quadratic constraint in formulation (2) below).

By introducing the auxiliary variables $y_{k}\in\mathbb{R}_{+},k=1,\dots,n$, the market assignment problem (1) can be reformulated as:

[TABLE]

This follows from the fact that for any optimal solution of (2), the constraints $y_{k}\geq(x_{k}-x_{k}^{0})^{2}$, $k=1,\dots,n$, will be tight. Now let $\mathcal{K}^{n+1}\subseteq\mathbb{R}^{n+1}$ denote the second-order (or Lorentz) cone (cf., Alizadeh and Goldfarb,, 2003) in dimension $n+1$; that is,

[TABLE]

where $\|\cdot\|_{2}$ represents the Euclidean norm. Note that like $\mathbb{R}^{n}$ or $\mathbb{R}^{n}_{+}$, the cone $\mathcal{K}^{n+1}$ is a closed convex cone (cf., Ben-Tal and Nemirovski,, 2001). Moreover, $\mathbb{R}^{n}$, $\mathbb{R}^{n}_{+}$, $\mathcal{K}^{n+1}$ are self-dual cones; that is, $(\mathbb{R}^{n})^{*}=\mathbb{R}^{n}$, $(\mathbb{R}^{n}_{+})^{*}=\mathbb{R}^{n}_{+}$, $(\mathcal{K}^{n+1})^{*}=\mathcal{K}^{n+1}$, where for any convex set, the dual cone of $S\subseteq\mathbb{R}^{n}$ is equal to $S^{*}=\{u\in\mathbb{R}^{n}:u^{\intercal}w\geq 0\text{ for all }w\in S\}$. Here, for any $u,w\in\mathbb{R}^{n}$, $u^{\intercal}w=\sum_{i=1}^{n}u_{i}w_{i}$ denotes the usual inner product in $\mathbb{R}^{n}$. The self-duality of $\mathbb{R}^{n}$, $\mathbb{R}^{n}_{+}$, and $\mathcal{K}^{n+1}$ will be key in deriving an appropriate dual problem associated with the continuous relaxation (i.e., when $z_{i}\in\{0,1\}$ is replaced by $z_{i}\in[0,1]$ for $k=1,\dots,n$) of problem (2).

From the definition (3), it is not difficult to see (cf., Lobo et al.,, 1997, eq. (7)) that for any $k=1,\dots,n$,

[TABLE]

Thus, problem (2) can be reformulated as the following mixed-integer second-order cone program (MISOCP) (Terlaky,, 2013):

[TABLE]

Like problem (4) above, a MISOCP is an optimization problem that besides a linear objective and linear constraints, has second-order cone constraints, as well as integer or binary variables. Problems with second-order cone constraints are widely used in applications in engineering and science. In particular, this type of constraints appear in structural design problems (Lobo et al.,, 1997, Sec. 3.5), electrical engineering (Lubin et al.,, 2016; Bienstock et al.,, 2014), healthcare (Mak et al.,, 2014), and supply chain management (Atamtürk et al.,, 2012). In many instances, this is a result of the need to take into account the uncertainty of problem parameters and obtain solutions that are robust; that is, perform well in different scenarios (see, e.g., Mak et al.,, 2014). Moreover the solution of MISOCP problems can be obtained using MISOCP solvers like MOSEK, CPLEX, and Gurobi.

Similar to O’Neill et al., (2005), we next use the optimal solution of (4) to obtain the shadow (dual) prices associated with the market-clearing and bidder operational constraints in (4). Namely, let

[TABLE]

that is, $z^{*}\in\{0,1\}^{n}$ is the vector of optimal values of the binary variables in (4). After replacing $z_{k}=z^{*}_{k}$, $k=1,\dots,n$ in (4), we obtain the following second-order conic program (SOCP) (cf., Lobo et al.,, 1997):

[TABLE]

Like problem (6) above, a SOCP is an optimization problem that besides a linear objective and linear constraints, has second-order cone constraints. The key characteristic of SOCPs that will be exploited here, is the fact that SOCPs are convex optimization problems. This follows from the fact that the second-order cone constraint $(w_{0},w)\in\mathcal{K}^{n+1}$ is a convex constraint. As a result, there is a rich duality theory (which generalizes linear programming duality) for these problems, as well as polynomial-time solution algorithms (cf., Alizadeh and Goldfarb,, 2003). In turn, these algorithms can be used together with branch & bound techniques (cf., Conforti et al.,, 2014) to solve MISOCP problems like (4).

Notice that in (6), we have associated the dual variables: $p_{0}$ with the market-clearing constraint, $q_{k}$ with the $k$-th bidder’s operation constraint, $p_{k}$ with the $k$-th bidder’s commitment constraint, and $(\gamma_{k},\alpha_{k},\beta_{k})$ with the $k$-th second-order constraint, for $k=1,\dots,n$. Also, for ease of notation, let $u\in\mathbb{R}^{n}$ represent the vector of variables $u_{k}$, $k=1,\dots,n$. With this notation (cf., Alizadeh and Goldfarb,, 2003), the dual SOCP corresponding to the primal SOCP problem (6) can be obtained by constructing the lagrangean dual of (6). Namely, let

[TABLE]

where

[TABLE]

for $k=1,\dots,n$, where $p_{0}\in(\mathbb{R})^{*}=\mathbb{R}$, $q\in(\mathbb{R}^{n}_{+})^{*}=\mathbb{R}^{n}_{+}$, $p\in(\mathbb{R}^{n})^{*}=\mathbb{R}^{n}$, and $(\gamma_{k},\alpha_{k},\beta_{k})\in(\mathcal{K}^{3})^{*}=\mathcal{K}^{3}$, for all $k=1,\dots,n$, are the dual variables or lagrangian multipliers associated to each of the constraints in problem (6).

From (7) and (8), it follows that the dual problem of (6):

[TABLE]

is equivalent to

[TABLE]

In Proposition 1 below, we show that strong duality holds between (6) and (9), and that their optimal objectives are attained. For this purpose, we introduce, for any set $S\subseteq\mathbb{R}^{n}$ the notion of its interior; that is, $\operatorname{int}(S)=\{w\in S:\text{ for any }u\in\mathbb{R}^{n},\text{ there exists }\epsilon>0,\text{ such that }w+\epsilon u\in S\}$.

Proposition 1**.**

Assume that (1) is feasible, and $a_{k}\neq 0$, $c_{k}\geq 0,r_{k}>0$, for all $k=1,\dots,n$. Then

[TABLE]

Proof.

From the discussion above, it is clear that problems (1), (4), and (6) are equivalent. Therefore, $p^{\ast}_{\operatorname{MIQP}}=p^{\ast}_{\operatorname{MISOCP}}=p^{\ast}_{\operatorname{SOCP}}$. It then remains to show both $p^{\ast}_{\operatorname{SOCP}}$ and $d^{\ast}_{\operatorname{SOCP}}$ are attained and strong duality holds between (6) and (9); that is, $p^{\ast}_{\operatorname{SOCP}}=d^{\ast}_{\operatorname{SOCP}}$. For this purpose, we first show that both (6) and (9) are strictly feasible (cf., Alizadeh and Goldfarb,, 2003).

Notice that from the feasibility of (1), the fact that the feasible set of (1) is closed, and that the objective of (1) is bounded below by $\min\{0,k\min_{k=1,\dots,n}\{d_{k}\}\}$, it follows from Weierstrass’ Theorem that (1) has an optimal solution. Let $x^{*}\in\mathbb{R}^{n}_{+}$, $z^{*}\in\{0,1\}^{n}$ be the optimal solution of (1), and consider any vector $y\in\mathbb{R}^{n}_{+}$ such that $y_{k}>(x_{k}^{*}-x_{k}^{0})^{2}$, for $k=1,\dots,n$. It is easy to see that $(x^{*},y,z^{*})\in\mathbb{R}^{2n}_{+}\times\{0,1\}^{n}$ is feasible to (6). Furthermore, we have that

[TABLE]

for $k=1,\dots,n$. Thus, $(x^{*},y,z^{*})\in\mathbb{R}^{2n}_{+}\times\{0,1\}^{n}$ is a strictly feasible solution for (6); that is, $(x^{*},y,z^{*})\in\mathbb{R}^{2n}_{+}\times\{0,1\}^{n}$ is feasible for (6), and $(y_{k}+1,y_{k}-1,2(x^{*}_{k}-x_{k}^{0}))\in\operatorname{int}(\mathcal{K}^{3})$, $k=1,\dots,n$. Thus, problem (6) is strictly feasible. Now consider the assignment

[TABLE]

for $k=1,\dots,n$. Clearly, (10) is feasible for (9), with

[TABLE]

for $k=1,\dots,n$. That is, (10) is a feasible solution for (9), and $(\gamma_{k},\alpha_{k},\beta_{k})\in\operatorname{int}(\mathcal{K}^{3})$, $k=1,\dots,n$. Thus, problem (9) is strictly feasible. The result then follows from SOCP duality (see, e.g., Alizadeh and Goldfarb,, 2003, Thm. 13). ∎

Now let us consider the individual problems associated to the bidders in the market. For each bidder $k=1,\dots,n$, let $t_{0}$ be the unit commodity price and $t_{k}$ be the price reflecting the commitment action offered to individual $k$ by the auctioneer. Define the following problem,

[TABLE]

which similar to problem (1) is equivalent to the following MISOCP:

[TABLE]

Similar to O’Neill et al., (2005), below we define both the market-clearing prices and associated market-clearing contracts between the auctioneer and the bidders.

Definition 1**.**

A competitive equilibrium for the market is a set of prices $\{t_{0}^{\ast},t_{k}^{\ast}\}$ and allocations $\{x_{k}^{\ast},z_{k}^{\ast}\}$, such that

1. (a)

At the prices $\{t_{0}^{\ast},t_{k}^{\ast}\}$, the allocations $\{x_{k}^{\ast},z_{k}^{\ast}\}$ solve (11) for all $k=1,\dots,n$; 2. (b)

The market clears: $\sum_{k=1}^{n}a_{k}x_{k}^{\ast}=b_{0}$.

Definition 2**.**

Let $T_{k}$ be a contract between the auctioneer and bidder $k\in\{1,\dots,n\}$ with the following terms:

1. (a)

Bidder $k$ operates following $z_{k}=z^{\ast}_{k},x_{k}=x_{k}^{\ast}$. 2. (b)

Bidder $k$ receives an amount from the auctioneer that is equal to the following payment: $p_{0}^{\ast}a_{k}x_{k}+p_{k}^{\ast}z_{k}$.

In what follows, we refer to $T=\{T_{k}\text{ for }k=1,\ldots,n\}$, to describe the market-clearing contracts between the auctioneer and the bidders. Below, we provide the main result of the article; namely, a characterization of the market-clearing prices for a market with non-convexities and convex quadratic costs.

Theorem 2.1**.**

Assume that (1) is feasible, and $a_{k}\neq 0$, $c_{k}\geq 0,r_{k}>0$, for all $k=1,\dots,n$. Let $\{x_{k}^{\ast},y_{k}^{\ast},z_{k}^{\ast}\}$ for all $k=1,\dots,n$ be an optimal solution to (1). Also, let $p_{0}^{\ast}$, and $\{p_{k}^{\ast},q_{k}^{\ast},\gamma_{k}^{\ast},\alpha_{k}^{\ast},\beta_{k}^{\ast}\}$ for all $k=1,\dots,n$ be an optimal solution to (9). If in (11) we define $t_{0}=p_{0}^{\ast}$ and $t_{k}=p_{k}^{\ast}$ for all $k=1,\dots,n$, then the prices $\{p_{0}^{\ast},p_{k}^{\ast}\}$ and allocations $\{x_{k}^{\ast},z_{k}^{\ast}\}$ for all $k=1,\dots,n$ represent a competitive equilibrium.

Proof.

Note that $\{x_{k}^{\ast},y_{k}^{\ast},z_{k}^{\ast}\}$ for all $k=1,\dots,n$ is also an optimal solution of (6). From the KKT conditions associated to the optimal solutions of both (6) and (9), it follows that:

[TABLE]

where the notation $u\bot w$, for $u,w\in\mathbb{R}$ denotes the complementary slackness between $u$ and $w$. Now consider the following problem under the contract $T$; that is, each individual bidder $k$ is offered prices $\{p_{0}^{\ast},p_{k}^{\ast}\}$, then each participant $k=1,\dots,n$ will solve (11) with $t_{0}=p_{0}^{*}$, $t_{k}=p_{k}^{*}$, $k=1,\dots,n$ to minimize their operation cost. That is, each bidder will solve

[TABLE]

Clearly, $(x_{k},y_{k},z_{k})=(x_{k}^{*},y_{k}^{*},z_{k}^{*})$ is feasible for (16), and the objective value of this solution, denoted $\hat{p}_{\operatorname{MISOCP}_{k}}(p_{0}^{*},p_{k}^{*})$, is

[TABLE]

Using the complementarity equations (13), (14), and (15), it follows that

[TABLE]

Next we show that $(x_{k},y_{k},z_{k})=(x_{k}^{*},y_{k}^{*},z_{k}^{*})$ is the optimal solution for problem (16). Let $(x_{k},y_{k},z_{k})\in\mathbb{R}^{2}_{+}\times\{0,1\}$ be a feasible solution of (16). It follows that $g_{k}x_{k}+h_{k}z_{k}-b_{k}\geq 0$, $y_{k}\geq 0$ and $(y_{k}+1,y_{k}-1,2(x_{k}-x_{k}^{0}))\in\mathcal{K}^{3}$. Therefore

[TABLE]

since the feasibility of $p_{0}^{\ast}$, and $\{p_{k}^{\ast},q_{k}^{\ast},\gamma_{k}^{\ast},\alpha_{k}^{\ast},\beta_{k}^{\ast}\}$ for all $k=1,\dots,n$, for (9) ensures that $q_{k}^{*}\geq 0$, $r_{k}-\gamma^{\ast}_{k}-\alpha^{\ast}_{k}\geq 0$, and $(\gamma_{k}^{*},\alpha_{k}^{*},\beta_{k}^{*})\in\mathcal{K}^{3}=(\mathcal{K}^{3})^{*}$. Also, from the fact that $p_{0}^{\ast}$, and $\{p_{k}^{\ast},q_{k}^{\ast},\gamma_{k}^{\ast},\alpha_{k}^{\ast},\beta_{k}^{\ast}\}$ for all $k=1,\dots,n$ is feasible for (9), $x_{k},z_{k}\in\mathbb{R}_{+}$, it follows that:

[TABLE]

Now let $p_{\operatorname{MISOCP}_{k}}$ denote the objective value of (16) associated with the feasible solution $(x_{k},y_{k},z_{k})\in\mathbb{R}^{2}_{+}\times\{0,1\}$; that is

[TABLE]

Using (17) and then (18), we have

[TABLE]

This shows that $(x_{k}^{\ast},y_{k}^{\ast},z_{k}^{\ast})$ is the optimal solution for (16). Furthermore, the solution $(x_{k}^{\ast},y_{k}^{\ast},z_{k}^{\ast})$ satisfies the market-clearing condition $\sum_{k=1}^{n}a_{k}x_{k}^{*}=b_{0}$, therefore, $(x_{k}^{\ast},y_{k}^{\ast},z_{k}^{\ast})$ will provide a market-clearing allocation. ∎

3 Scarf’s market instance

As an example of a market with non-convexities, consider a problem proposed by Scarf, (1994). The objective is to minimize the total cost while satisfying the demand in an electricity market. Two types of plants are available to provide the electricity in the market. The characteristics of each type of plant, including costs and operational constraints are summarized in Table 3.1.

Scarf’s market problem can be formulated as the following mixed-integer linear program

[TABLE]

where $D$ is the total demand. The market-clearing price for this problem has been studied in O’Neill et al., (2005). Table 3.2 summerizes the optimal solution of (19) for different values of the demand. It is clear that as the demand increases, the number of plants of different types used can change dramatically. For example, when the demand is 56, all type 1 plants are closed and eight (8) type 2 plants are open; however, when the demand is 60, two (2) type 1 plants are open while now only four (4) type 2 plants are open. The market-clearing prices in Table 3.3 are obtained from the dual (shadow) prices of the linear program obtained from (19) by fixing its binary variables to their optimal value (cf., O’Neill et al.,, 2005). Note that for all the instances with different demand, the market-clearing prices remain the same.

Now we consider a modified Scarf problem with quadratic ramping costs,

[TABLE]

where $x_{1i}^{0},x_{2j}^{0}$, for all $i=1,\dots,5$, $j=1,\dots,10$, are set to be the optimal unit’s generation obtained after solving (19) with a demand $D=55$. The quadratic terms in the objective function can be interpreted as the ramping costs of moving from the original output plan set from (19) with a demand $D=55$. It is clear that if we set $D=55$ in (20), then no matter what the values of $r_{1},r_{2}$ are, problem (20) will have the same optimal solution.

After setting $r_{1}=r_{2}=0.1$, we can see from Table 3.4 that in most cases, the number of type 1 plants and type 2 plants with full capacity remain the same as a result of the ramping costs (in contrast with Table 3.2). Note that as the demand increases, a type 2 plant with partial capacity is opened for production. Table 3.5 summarizes the market-clearing prices obtained using the results in Section 2. In order to maintain a competitive equilibrium, the market-clearing prices vary from case to case. In contrast with the start-up prices obtained by O’Neill et al., (2005), note that from the results of Table 3.5, it follows that the start-up price can differ for type 2 plants producing at full capacity, and type 2 plants producing at partial capacity. Specifically, the start-up price of type 2 plants producing at full capacity changes with the demand, while the start-up price of type-2 plants producing at partial capacity remains the same for demands between 56 to 68. The start-up price for closed type 2 plants is the same as the price for type 2 plants with full capacity. The unit commodity price varies in a small range but it does not remain the same. The start-up price for all type 1 plants remains mostly equal to 53, but this price is different for some demand levels (e.g., compare $D=60$ and $D=62$).

Tables 3.6 and Table 3.7 show that after setting $r_{1}=0.1,r_{2}=0.3$ in (20), similar conclusions as in the case $r_{1}=r_{2}=0.1$ can be reached. However, with a higher ramping cost on type 2 plants, we can see that with demand values between 64 to 68, instead of operating one more type 2 plant to satisfy the demand, the optimal solution suggests to operate an additional type 1 plant instead.

Figure 3.1 and Figure 3.2 graphically compare the solutions obtained from the three cases discussed thus far; that is when $r_{1}=r_{2}=0$, when $r_{1}=r_{2}=0.1$, and when $r_{1}=0.1,r_{2}=0.3$.

Next we consider an extreme case where the parameter of ramping cost is relatively large. With $r_{1}=r_{2}=1$, similar analysis can be applied from the analysis with $r_{1}=r_{2}=0.1$. However, with a relative large ramping cost parameter, the unit commodity price as well as start-up price can change dramatically. It is interesting to see that when the demand becomes 58, the start-up prices for type 1 plant, type 2 plant with full capacity and closed type 2 plant are negative, which means these plants need to pay instead of getting paid to open in exchange for a very high commodity price (compared with Table 3.3 and Table 3.5).

Now consider the case in which $r_{1}=r_{2}=1$ and $D=60$ in (20), as an example to illustrate that the dual prices obtained by using the methodology of Section 2, result in a competitive equilibrium. For any $j=1,\dots,n$, the individual problem for a type 2 plant $j$ committed by the central operator to produce at partial capacity is

[TABLE]

and its optimal solution is $(x^{\ast}_{2j},z^{\ast}_{2j})=(5,1)$, which matches the optimal solution in Table 3.8. It is not difficult to check, given the dual prices in Table 3.9, that the optimal solution for every individual problem matches its corresponding solution in Table 3.8, verifying Theorem 2.1.

4 Conclusion

In this paper, we considered the problem of obtaining appropriate market-clearing prices when the market has both non-convexities and convex quadratic costs. Our results show that by using convex optimization techniques, the seminal work of O’Neill et al., (2005) on pricing in markets with non-convexity, can be extended to markets in which both non-convexities and convex quadratic costs arise. In the electrical market these two market features arise due to generator fixed costs (or other operational constraints), and ramping or quadratic generation costs. Considering both of these features in the electricity market have become increasingly important due the high penetration of renewable energy sources (RES). This is due to the output volatility of RES, which requires conventional generators to ramp up or down more frequently. Besides electricity markets non-convexity features appear in other markets such as the financial and labor market (Brown and Smith,, 2011; Bentolila and Saint-Paul,, 1992). Thus our results have an impact in a wide range of potential markets.

Furthermore, we believe that the techniques outlined here can be used to extend other pricing methodologies for markets with non-convexities (Liberopoulos and Andrianesis,, 2016, see,) that aim to obtain prices with different characteristics. Finally, in the context of electricity markets, it is natural to consider how the consideration of ramping costs will affect prices in an electrical network with congested lines. Addressing these questions provides interesting directions for future research work in this area.