Robust capacity expansion modeling for renewable energy systems

TL;DR

This paper introduces a method to plan renewable energy systems that account for weather variability, ensuring reliable supply while keeping costs low.

Contribution

A novel algorithm for robust capacity expansion planning that iteratively adjusts for weather uncertainty in renewable energy systems.

Findings

Robust systems increase annual costs by 1.6-2.9%, but prevent supply gaps during extreme weather.

Non-robust systems can face up to 50% loss of load during atypical weather periods.

Using average weather years and iteratively adjusting solutions is a best practice for reliable modeling.

Abstract

Future greenhouse gas neutral energy systems will be dominated by renewable energy technologies providing variable supply subject to uncertain weather conditions. For this setting, we propose an algorithm for capacity expansion planning: We evaluate solutions optimized on a single years’ data under different input weather years, and iteratively modify solutions whenever supply gaps are detected. These modifications lead to solutions with sufficient capacities to overcome periods of cold dark lulls and seasonal demand/supply fluctuations. A computational study on a German energy system model for 40 operating years shows that preventing supply gaps, i.e., finding a robust system, increases the total annual cost by 1.6-2.9%. In comparison, non-robust systems display loss of load close to 50% of total demand during some periods. Results underline the importance of assessing the feasibility of energy system models using atypical time-series, combining dark lull and cold period effects.

•In energy system modeling, disregarding weather variability leads to misinvestment•Average weather years fail to capture intra-/inter-year variability•Best practice: Use average years, but validate and iteratively adjust solutions•Protection against weather variability requires only 2%–3% additional annual costs

In energy system modeling, disregarding weather variability leads to misinvestment

Average weather years fail to capture intra-/inter-year variability

Best practice: Use average years, but validate and iteratively adjust solutions

Protection against weather variability requires only 2%–3% additional annual costs

Applied sciences

Figures

Flowchart depicting the proposed methodology for determining robust energy systemsIn each main loop iteration, $[eqn]$ unit commitment problems (UC) and one modified capacity expansion problem ( $[eqn]$ ) are solved until loss of load is sufficiently small.

TAC comparison by technology for ESMs optimized from 1980 to 2019 aggregated for a 38 node Germany model set up in ETHOS.FINE

TAC comparison by technology for ESMs optimized from 1980 – 2019 on a single node Germany model set up using *gurobipy*

Load shedding when testing the feasibility of the 5 selected reference years in ETHOS.FINEOnly the years and the months from October to March are included, where load shedding occurs. For comparison, hourly load in the demand data, we use roughly 100 GW.

Six time periods from the 40 weather years and 1 year of future electricity demand data for GermanyThe electricity demand is normalized to prevent overweighing; the weather data are aggregated The upper three diagrams represent non-critical time periods, the lower three critical ones. Subfigure (A) is a typical summer period with high PV availability and low electricity demand due to low heating requirements. In (B) and (C), typical autumn and winter period are shown. They are characterized by low availability of PV, but ample wind power to supply sufficient electricity. Note the increased ele

TAC comparison from $[eqn]$ for robust solutions using modification smoothed MOD 1/Demand increase for the single node model in *gurobipy*, no temporal aggregation

TAC comparison for optimizing for from $[eqn]$ for robust solutions using modification *MOD 2/Synthetic time-seriesgurobipy* for the singe node model in *gurobipy*, no temporal aggregationFigure 8TAC comparison for optimizing for 5 selected reference years for Germany Columns marked with an ∗ indicate the robust system designed via *MOD 2/Synthetic time-series* ETHOS.FINETable 5Capacity results of optimizing for 5 selected years for Germany. Columns marked with an ∗ indicate the robust system designed with *MOD 2/Synthetic time-series*Capacities19871987∗19961996∗20092009∗20102010∗20132013∗Wind

Feasibility testing of the energy system optimized for 1987 in 1994 before modification on the left and after modification on the rightThe cold dark lull period in early February (see [Figure 5](#fig5)F) is marked in red. In the left graphic, due to insufficient backup capacity the supply gap variable has to be utilized meaning the energy system is not robust, i.e., there are still supply gaps after optimization.In the right graphic, the energy is fully operational during the cold dark lull after applying MOD 2/Synthetic time-series to the original optimization problem.

TAC comparison for robust solutions based on single years using MOD 3/Combine, a single node *gurobipy* model

The left graphic shows wind on- and offshore as well as PV full load hours compared to TAC in for all yearsThe right graphic shows combined cost for wind and for PV as share of the TAC compared to TAC for all yearsEach dot represents 1 year. Lines indicate regression lines of the different energy generator types.

Full text

Introduction

Modeling energy systems require access to different types of data. For future greenhouse gas neutral energy systems, time-series data are of special importance, as a vast expansion of wind power and solar photovoltaic (PV) that are dependent on the weather is perceived as indispensable.1^,^2 However, we cannot expect to have accurate long-term information on future weather conditions. Instead, historical time-series serve as a substitute in capacity expansion planning, and often a single “representative” sample year is selected for optimization.3 For example, the International Renewable Energy Agency (IRENA) recommends using 2018 as a reference year because it represents generation from renewable technologies well on average.4 In the following, we argue that using “representative” sample years may lead to energy system models (ESMs) that appear sound at first glance, but would fail to meet supply in reality. We then propose a tractable approach to counteract this effect and to make energy systems robust against uncertain conditions during operation, i.e., ensure security of supply.

There is ample evidence in literature that the results of energy system optimization are sensitive to changes in weather time-series data. Schyska et al. (2021) evaluated the sensitivity of capacity expansion models with regards to multiple sources of uncertainty. They conclude that some years are unsuited as reference years, as using them for optimization leads to significant misallocation of assets.5 Ruggles et al. (2024) assessed how many years of weather data are needed to ensure ESMs are reliable even out of sample, i.e., if a different weather year were to realize. They conclude that between 15 and 40 years are required depending on amount of imports available/the acceptable level of loss of load.6 The works by Haddeland et al.,7 and Staffel et al.,8 which builds on earlier work by Pfenninger et al.,9 similarly find the choice of weather years significantly effects renewable generation and thus power output. Not only that, but also the effect of the weather increases with increasing share of renewable technologies.10 De Marco et al.11 identified energy shortages across Europe and use those to optimize climate-resilient energy systems stochastically. Not all research agrees on this; Schlachtenberger et al.12 optimized three weather years with hourly data both individually and as one time-series with a resolution of 3 h per time step, finding only small variations in total annual cost (TAC) and installed capacities. However, they note that aggregating multiple hours together introduces a smoothing effect that systematically favors PV and underestimate battery and wind generation requirements.

Additionally, there is a fundamental tension between identifying typical years, and build ESMs that are protected against extreme events, i.e., robust. On this, Hilbers et al.13 introduced a method of importance subsampling for time-series aggregation to explicitly preserve extreme events in the weather data as an alternative to established “representative days” clustering approaches. Ryberg’s dissertation14 and Ryberg et al.15 investigated the impact of generation lulls in an energy system for a large part of Europe calculating backup capacities required to overcome these. Ruhnau et al.16 looked into the storage requirements for a renewable-based ESM for Germany using 35 years of weather data taking consecutive extreme events into account. They concluded that consecutive extreme events increase storage requirements significantly compared to even the most extreme, but singular events. Thus, atypical weather time-series may be particularity well-suited for optimization because they capture important system behaviors, e.g., dark lulls, that significantly impact ESMs. In this context, Grochowicz et al.17 discussed optimizing sequential weather years. They use a geometry-based solution approach targeting the solution space. In follow-up work, they use electricity shadow prices to identify difficult weather periods.18 They observe that such difficult weather periods are not just meteorological events, but results of the interplay of meteorology and electricity storage and network structures.

In stochastic optimization, the notion of using typical, i.e., expected, behavior as a baseline for optimization is a well-known concept. Its usefulness is determined by the value of a stochastic solution, which represents the gap in expected performance of a solution obtained with expected data and one determined by solving a full stochastic optimization problem.19 In general, this gap may be arbitrarily large. However, research has also shown that we can still extract insights from solutions obtained with such data.20 Specifically concerning weather robustness, Forghani et al. (2025) proposed an intermediate approach where not only a representative but also worst-/best-cost years are used as input data for a stochastic optimization. Their results show that this reduces loss-of-load to practically zero, at $[eqn]$ additional cost.21

An important modeling decision in this context is the value and availability of recourse, i.e., the ability to specify or change parts of the solution if uncertainty realizes. In capacity expansion planning, imports constitute such recourse. If arbitrary imports are allowed, any misplanning can be compensated for, if at a cost. We focus on a different setting, the one where imports are limited or energy systems are to be self-sufficient. Here, we use adaptive robust optimization (ARO) to model limited recourse while still enforcing strict guarantees on security of supply general solution approaches. To the best of our knowledge, Zeyringer et al.22 were the first to assess the effects of weather uncertainty in input data on capacity-expansion planning, looking at an ESM of Grein. They proposed using multiple historical time series, optimizing over each, and then evaluating the operational costs/supply costs incurred by the proposed energy systems under the assumption that the time-series of a different historical year realizes. Then, an ESM with lowest worst-case costs/lowest supply gap among the solutions is selected. They find that starting with the “wrong” reference year may lead to misallocation of resources and that modeling with multiple years leads to more consistent results with lower worst-case cost, the latter of which had also been noted before.23 For the United States, Dowling et al.24 make the case for multi-year modeling to accurately capture long-term storage effects, noting that the cost of variable renewable power systems are especially sensitive to long-duration storage costs. A recent work by Gøtske et al.25 also assessed energy systems based on different weather years. They employed $[eqn]$ emitting backup technologies and analyzed structural elements of the respective solutions. Either approach allows to select more suitable ESMs, but it cannot assure solutions meet certain supply/demand across all years.

The two-stage robust setting

To illustrate the setting considered in this work, we begin with an example for an ARO ESM. ARO, alternatively called two-stage robustness, is characterized though a bilevel structure26 in which a decision maker has to make a set of first stage decisions $[eqn]$ , then one of multiple scenarios $[eqn]$ may realize and afterward the decision maker has the option to react to scenario $[eqn]$ with a scenario-dependent set of decisions $[eqn]$ . Equation (ARO) illustrates the structure of an ARO problem. Here, $[eqn]$ and $[eqn]$ are vectors of first and second stage costs, respectively. The matrix $[eqn]$ encodes all information that is not affected by uncertainty, whereas the matrices $[eqn]$ and the right-hand side vector $[eqn]$ are dependent on the scenario $[eqn]$ .

$$[eqn]$$

For the ESM(s) considered in this work, the first stage decisions correspond to the capacity expansion planning, e.g., investment in solar power plants, power lines, storage units, and the second stage corresponds to operational decisions, e.g., how much energy is to be stored in which battery at what time. If energy imports are allowed, these will also be encoded in $[eqn]$ . The scenario $[eqn]$ corresponds to a specific, continuous time period. Table 1 illustrates this setting. Here, we consider different locations $[eqn]$ for a solar power plant, whose production is subject to changing weather patterns during the time periods $[eqn]$ .Table 1. Investment problem: Chose number of solar power plants to build on locations $[eqn]$ ScenarioLocation ALocation BLocation C $[eqn]$ 2007550 $[eqn]$ 5075150Values indicate expected energy supply in $[eqn]$ per investment.

The optimal solution depends on what type of recourse is available and how much supply is needed. For example, if we assume similar cost at all locations and no/expensive imports, investing in location $[eqn]$ is the most cost effective for meeting a demand of $[eqn]$ . However, if the total demand were $[eqn]$ , it is more effective to invest in $[eqn]$ and $[eqn]$ instead of $[eqn]$ .

Returning to our example in Table 1, the approach by Zeyringer et al. would not be able to find the optimal robust solution for a total demand of 75 MWh/year, which is to invest in location $[eqn]$ . This is because if the future is known, i.e., we assume either $[eqn]$ or $[eqn]$ to realize, the optimal investment is always to either invest in location $[eqn]$ or location $[eqn]$ . An alternative approach was put forward by Caglayan et al. (2019),27 who noted variations of up to $[eqn]$ variations in TAC, and significant differences in energy system designs when optimizing an integrated electrical/hydrogen ESM for Western Europe with different weather years. They propose an iterative approach, where they begin with optimizing individual years, but then average their designs and take the average as a lower bound for installed capacities in a new round of optimization.28 Since the (average) capacities installed during sequential algorithm iterations will be non-decreasing and bounded, convergence and thus a feasible solution will be guaranteed in most cases. In our example, the algorithm would enforce capacity $[eqn]$ for location A and C, which would eventually lead to a feasible but unnecessarily expensive solution that uses both A and C.

A novel algorithm for computing robust ESMs

The proposed algorithm consists of four main steps, as shown in Figure 1. As its input, the algorithm takes a set of $[eqn]$ times series, one of which is designated as a starting/reference year. We begin with solving the capacity expansion planning problem for this year (CAPEX), then we evaluate the performance of the proposed investment in all $[eqn]$ years by solving the corresponding unit commitment problem (UC), which formalizes the validation proposed in previous works.22^,^25Figure 1. Flowchart depicting the proposed methodology for determining robust energy systemsIn each main loop iteration, $[eqn]$ unit commitment problems (UC) and one modified capacity expansion problem ( $[eqn]$ ) are solved until loss of load is sufficiently small.

To ensure feasibility, existing approaches allow for energy imports. We consider the robust setting where imports are bounded/not available, i.e., the ESM needs to be self-sufficient. Based on the previous step, we identify supply gaps, which we denote by $[eqn]$ . The underlying ESM for Germany for 2045 utilizes renewable technologies only and includes PV, wind, Li-ion batteries, electrolyzer, hydrogen salt caverns, hydrogen combined cycle gas turbines as well as electricity transmission and hydrogen pipelines. The models are described in detail in the STAR Methods section for both gurobipy and the ETHOS.FINE. If the supply gaps are sufficiently small, we terminate. Otherwise, we use the information encoded in $[eqn]$ to iteratively make solutions more robust. For that, modifications (MOD) are applied to the optimization problem by either adding artificial demand MOD 1/Demand increase, substituting parts of the time series data MOD 2/Synthetic time-series or iteratively adding both demands and lazy valid inequalities MOD 3/Combine, before reoptimizing CAPEX to evaluate the loss of load. Detailed definition of the modifications can be found in the methods section.

The main contributions of our work include not only insights derived for the specific ESM, and an algorithm for solving CAPEX problems to find solutions that are robust to variations in operational conditions across the $[eqn]$ years, but foremost a systematic comparison of different approaches for designing robust ESMs. Based on this, we recommend three working modifications that all lead to robust energy systems.

Results and discussion

In the following, we first compare solutions derived from individual years’ data and assess their (lack of) robustness. Then, we modify them to become more robust, and evaluate the features of robust solutions. Note that we provide results for two different models, since some modifications require custom callbacks/lazy constraints which are hard to integrate into modeling frameworks. Thus, we use one model directly coded in gurobipy that allows for all modifications but is simplified, see reference linear program, and a more realistic model implemented in the ETHOS.FINE framework. Table 2 provides an overview over the technologies considered and Table 3 provides the techno-economic parameters used in this study.Table 2. Energy system components considered for development of the proposed methodology for optimising ESMs for GermanyTechnologySupplyRooftop PV, Open field PV, Onshore wind, Offshore windStorageLi–ion batteries, H_2_ salt cavernsTransmissionElectricity grid, H_2_ pipelinesConversionElectrolysers, H_2_ combined cycle gas turbines (CCGT)DemandElectricity demandTable 3Techno–economic parameters considered in this workTechnology $[eqn]$ $[eqn]$ Life time [a]SourcePV (Rooftop)474 $[eqn]$ 10 $[eqn]$ 20Tsiropoulos et al.53 and Kelm et al.54PV (Open field)320 $[eqn]$ 5.4 $[eqn]$ 20Tsiropoulos et al.53Wind (Onshore)1000 $[eqn]$ 25 $[eqn]$ 20Tsiropoulos et al.53 and Kreidelmeyer et al.55Wind (Offshore)2530 $[eqn]$ 63 $[eqn]$ 20Robinius et al.56Li–ion batteries131 $[eqn]$ 3.3 $[eqn]$ 15Stolten et al.57H_2_ salt caverns0.7 $[eqn]$ 0.01 $[eqn]$ 40Caglayan et al.58Electricity grid0.86 $[eqn]$ 0.03 $[eqn]$ 40Etri59H_2_ pipelines0.185 $[eqn]$ 0.01 $[eqn]$ 40Caglayan et al.28Electrolysers350 $[eqn]$ 11 $[eqn]$ 10Stolten et al.57CCGT hydrogen gas760 $[eqn]$ 23 $[eqn]$ 20Stolten et al.57Additionally, in ETHOS.FINE we also assumed a self discharge of $[eqn]$ per hour for Li–ion batteries.

Note that we also only use a single demand time-series from 2012 for comparison. This significantly simplifies evaluation of the algorithms as otherwise, loss of load and installed would need to be considered relative to each year’s demands. However, in most practical settings where demands and weather are linked, e.g., when employing significant electric or hydrogen-based heating, the paired demand time-series ought to be used.

Optimal capacities strongly depend on yearly weather data

The TAC for energy systems for the 38 node model of Germany within ETHOS.FINE for the 40 different years of time-series data deviates around an average of 106bn€ with cost between 96.4bn€ and 113.6bn€ annually, which equals $[eqn]$ to $[eqn]$ compared to the default reference year 2018 recommended by IRENA.4 The results of optimizing each year independently are given by Figure 2. While the variations in overall TAC are limited, the energy system designs show substantial deviations. The cost shares of single technologies across the 40 different single years vary by $[eqn]$ for hydrogen pipelines, $[eqn]$ for hydrogen salt caverns, $[eqn]$ for CCGT, $[eqn]$ for Li-ion batteries, $[eqn]$ for PV, $[eqn]$ for electrolysers, $[eqn]$ for the electricity grid, and $[eqn]$ for onshore wind, making it challenging to draw recommendations for planned capacity expansion for future energy systems.Figure 2TAC comparison by technology for ESMs optimized from 1980 to 2019 aggregated for a 38 node Germany model set up in ETHOS.FINE

In the simplified single node model, the cost variation for single technologies is on average slightly higher than in the 38-node model: onshore wind $[eqn]$ , rooftop PV $[eqn]$ , Li-ion batteries $[eqn]$ , hydrogen salt caverns $[eqn]$ , electrolysers $[eqn]$ , and CCGT $[eqn]$ . Open field PV is installed up to its maximum capacity for all years, while offshore wind is never utilized. A plot of all individual years is given in Figure 3.Figure 3TAC comparison by technology for ESMs optimized from 1980 – 2019 on a single node Germany model set up using gurobipy

In summary, even small input data differences may lead to large variations in investment into different technologies. For example, offshore wind is not utilized at all in some years, but makes up over $[eqn]$ of the TAC in 2014. Note that the single node model is not directly comparable to the ETHOS.FINE model, as it contains various simplifications. As such, solutions are significantly cheaper averaging 55.4bn€ with between 52.1 and 59.7bn€ annually, i.e., $[eqn]$ to $[eqn]$ .

Systems optimized on single years cause supply gaps

Feasibility testing shows that all 40 energy system designs solely based on one year’s time-series lead to supply gaps in multiple other years. This means that none of the system designs are robust. However, we are not only interested in whether there is any loss of load but also in the magnitude of said loss of load. Figure 4 shows the amount of load shedding when testing the feasibility of one of five reference years used in ETHOS.FINE during the other 39 years.Figure 4. Load shedding when testing the feasibility of the 5 selected reference years in ETHOS.FINEOnly the years and the months from October to March are included, where load shedding occurs. For comparison, hourly load in the demand data, we use roughly 100 GW.

Two trends can be observed: On the one hand, when the feasibility is tested for 1987 and 2010, load shedding occurs mostly in the first half of February. This is caused by demand spikes in early February due to the low temperatures in that time of the year, which the electricity demand was based on. At the same time, the energy systems optimized for 1987 and 2010 are characterized by the highest amounts of PV and wind capacities installed while having insufficient backup capacities.

On the other hand, the energy system of the reference years 2009 has increased backup capacities, therefore suffering less during cold dark lull periods, but does not have enough PV and wind capacities to produce enough hydrogen and therefore has load shedding more evenly distributed over the months from October to March. The other two reference years 1996 and 2013 are in the middle of these two cases.

Figure 5 illustrates the reasons for that by showing three uncritical (top row) periods with sufficient supply as well as three critical (bottom row) time periods with significant supply gaps. During critical time periods electricity generation from PV and wind is low, providing less than 50% of electricity demand. The remaining electricity demand needs to be fulfilled by backup capacities, i.e., $[eqn]$ CCGT. The periods have varying duration and they are identified via clustering and feasibility testing. Uncritical time periods are characterized by high availability of PV or onshore wind or both, while critical ones are characterized by low availability of PV and low to negligible onshore wind combined with high demand. Offshore wind plays only a minor role due to its limited utilization. The most critical time period takes place in 1994 (see Figure 5F). Here, wind and solar supply indicate a dark lull. Combined with the cold period identified in the electricity data, this constitutes a cold dark lull.Figure 5. Six time periods from the 40 weather years and 1 year of future electricity demand data for GermanyThe electricity demand is normalized to prevent overweighing; the weather data are aggregated The upper three diagrams represent non-critical time periods, the lower three critical ones. Subfigure (A) is a typical summer period with high PV availability and low electricity demand due to low heating requirements. In (B) and (C), typical autumn and winter period are shown. They are characterized by low availability of PV, but ample wind power to supply sufficient electricity. Note the increased electricity demand due to increased heating required. In (D), a typical dark lull is characterized by low availability of PV and negligible amounts of wind, which coincides with high electricity demand due to increased heating. Subfigure (E) shows an elongated dark lull period. Low availability of both PV and wind combined with increased electricity demand lead to overall difficult period requiring large amounts of hydrogen to be burned in the energy system. The last graphic (F) shows the most critical period in the 40 years of data. Negligible wind combined with low availability of PV coincide with the highest electricity demand in the data due to high heating demand during an extreme cold spell in all of Germany.

For the gurobipy implementation, we get similar results: The minimal annual supply gap across all years is 1.170 GWh for 1987, which is about $[eqn]$ of the total annual electricity demand. Additionally, during shorter time-periods the supply gap can reach up to half of the required electricity demand. In comparison, optimizing using 1990s weather data leads to a peak supply gap of 69.500 GWh, i.e., $[eqn]$ , if the weather of 1987 were to realize.

In summary, this strongly supports the results by Cagalayan et al.27^,^28—weather patterns matter and ignoring in energy systems with significant renewables may lead to systems that have large supply gaps under anything but the most optimal conditions, which we cannot simply assume to be covered through imports.

Effect of modifications on installed capacities

All modifications lead to increased investment in CCGT, salt caverns and electrolysers, although the latter is less pronounced for MOD 1/Demand increase, as seen in Table 4. This is expected, as CCGT can provide electricity when the generation is low and demand is high and installing more CCGT also requires more $[eqn]$ infrastructure such as salt caverns for storage and electrolysers for $[eqn]$ conversion.Table 4. Comparison of the different modifications in terms of convergence and performanceModificationIncreased InvestmentAverageLeastMOD 1/Demand increaseLi-Ion, $[eqn]$ -CCGT, Salt caverns+5.7%+2.0%Smoothed MOD 1/Demand increase $[eqn]$ -CCGT, Salt caverns+9.2%+2.4%MOD 2/Synthetic time-series**gurobipyPV, $[eqn]$ -CCGT, Salt caverns, $[eqn]$ -Electrolysers+2.5%+1.6%MOD 2/Synthetic time-series ETHOS.FINEPV, $[eqn]$ -CCGT, Salt caverns, $[eqn]$ -Electrolysers+3.7%+2.9%MOD 3/CombinePV, Wind, $[eqn]$ -CCGT, Salt caverns, $[eqn]$ -Electrolysers+10.6%+2.5%Average and least cost increase compared to best lower (dual) bound, the highest TAC of a single unmodified year.

Note that the more detailed model implemented in ETHOS.FINE was used for evaluation of MOD 2/Synthetic time-series, since it outperformed the other algorithms when implemented in gurobipy and is therefore viewed as the most promising approach. An additional advantage of MOD 2/Synthetic time-series: It can be easily integrated into existing modeling frameworks.

MOD 1/Demand increase leads to robust energy systems regardless of the initial time-series chosen. On average, this incurs additional cost of 7.8bn€ if no smoothing is performed. The average total cost of a robust system reaches 63.1bn€, with a range of [60.9,69.7]bn€. In comparison, using smoothing leads to slightly more expensive solutions. On average, making an ESM robust incurs additional cost of 9.7bn€(+ $[eqn]$ ). The average total cost of a robust system reaches 65.2bn€, with a range of $[eqn]$ bn€.

Figure 6 gives the results of optimizing each year independently for the smoothed MOD 1/Demand increase. While on average more expansive than non-smoothed MOD 1/Demand increase, the smoothed algorithm leads to on average lower investment increase in short-term battery storage ( $[eqn]$ non-smoothed vs. $[eqn]$ for smoothed) and a significantly higher investment in CCGT ( $[eqn]$ for non-smoothed vs. $[eqn]$ for smoothed). The higher costs may be due to the fact that additional artificial demand is added in time periods adjacent to those with previous supply gaps, which generates small supply demand gap time periods. The strong invest in CCGT compared to the non-smoothed modification suggests that CCGT power plants are used to offset those artificial demand gap time periods. For a visual comparison of the smoothing effect compared to non-smoothed we refer to Figure 12 in the appendix.Figure 6TAC comparison from $[eqn]$ for robust solutions using modification smoothed MOD 1/Demand increase for the single node model in gurobipy, no temporal aggregation

In summary, smoothed MOD 1/Demand increase leads to feasible solutions, but care should be taken that artificial energy demands do not lead to excessive building of CCGT power plants. Convergence of MOD 1/Demand increase can be slow, sometimes taking more than 20 iterations for a single pair of years. This number of iterations was used as a cut-off criterion, as no improvements were observed after that during testing. This appears to be due to very small residual supply gaps of a few $[eqn]$ that get found and added to the model repeatedly. Given the small size of those supply gaps, the large overall production, and the fact that Gurobi29 was used as a solver, which does not perform exact arithmetic, this may be caused by numerical instabilities. Using a suitable termination criterion (e.g., number of iterations or total supply gap less than some small number of GWh) counteracts this.

Notably, non-smoothed MOD 1/Demand increase incurs a bias toward installing more Li-ion battery storage. This is to expected, as artificial short term demand peaks are added, and Li-ion batteries are well-suited to compensate for those. Their capacity was increased by on average more than $[eqn]$ , with a range of [7.7,19.1]bn€, compared to [6.5,10.8]bn€ in the reference years. Finally, non-smoothed MOD 1/Demand increase finds the overall cheapest robust solution. That solution is characterized by slightly more investment in onshore wind capacity (19.8bn€, $[eqn]$ ) and roof top PV (10.2bn€, $[eqn]$ ) than in an average single year solution. No additional batteries are installed, but more electrolysers (5.6bn€, $[eqn]$ ), CGGT (7.3bn€, $[eqn]$ ), and salt caverns (2.7bn€, $[eqn]$ ).

MOD 2/Synthetic time-series is evaluated in Figures 7 and 8 for the model in gurobipy and the model in ETHOS.FINE, respectively, and Table 5, which give an overview of the results of generating robust ESMs. In the ETHOS.FINE model the five most expensive solutions based on the single years $[eqn]$ , and 2013 were selected of the 40 energy system designs to make them robust using MOD 2/Synthetic time-series. These are hereon referred to as the five reference years.Figure 7TAC comparison for optimizing for from $[eqn]$ for robust solutions using modification MOD 2/Synthetic time-series**gurobipy for the singe node model in gurobipy, no temporal aggregationFigure 8TAC comparison for optimizing for 5 selected reference years for Germany Columns marked with an ∗ indicate the robust system designed via MOD 2/Synthetic time-series ETHOS.FINETable 5Capacity results of optimizing for 5 selected years for Germany. Columns marked with an ∗ indicate the robust system designed with MOD 2/Synthetic time-seriesCapacities19871987∗19961996∗20092009∗20102010∗20132013∗Wind (onshore) $[eqn]$ 258243217189259241244241239202Wind (offshore) $[eqn]$ 0.00.00.21.80.00.00.41.412.417.7PV (rooftop) $[eqn]$ 400427430505356411394420374423PV (open field) $[eqn]$ 348348348348348348348348348348Li–ion batteries $[eqn]$ 722840742888684770770765635802H_2_ salt caverns $[eqn]$ 195211191234202237204217169206CCGT hydrogen gas $[eqn]$ 10112111211812412599125108121Electrolysers $[eqn]$ 142140144165142161137146137148Electricity grid $[eqn]$ 444444429405386386405425483463Hydrogen pipelines $[eqn]$ 9148008896861029686800686686571

On the one hand, applying MOD 2/Synthetic time-series to the gurobipy model leads to robust solutions with the lowest cost increase on average. As can be seen in Figure 7, the TAC is similar across all 40 years. On average, additional cost of 5.6bn€ or $[eqn]$ with a range of $[eqn]$ are the result of this modification. The overall deviation from the average robust solution is in the range $[eqn]$ showing the effectiveness of the algorithm independent of the selected reference year. The consistently lower costs indicate that MOD 2/Synthetic time-series is the best performing modification.

On the other hand, applying MOD 2/Synthetic time-series in ETHOS.FINE is shown in Figure 8. After modifying the five selected reference years with MOD 2/Synthetic time-series to make the solutions robust, the share of TAC for wind onshore decreases $[eqn]$ . Similarly, a decrease in total investment in transmission (electricity grid and hydrogen pipeline) is observed $[eqn]$ . A general increase in investment is seen for PV $[eqn]$ , Li–ion batteries $[eqn]$ as well as the hydrogen sector $[eqn]$ for the robust energy system designs. The increase of PV can be explained by its below average, but still relevant, availability during dark lulls combined with Li-ion batteries to cover daily fluctuations. As visible in Figure 9, PV is mainly utilized together with Li-ion batteries to cover the fluctuating part of the electricity demand, the CCGT cover the bulk of the electricity demand, while the generation from wind is negligible. Hydrogen is utilized for electricity generation to a higher degree, since it can provide flexible additional energy supply, especially during dark lulls. The overall increase in cost compared to the average cost of each of the five reference years is $[eqn]$ , compared to the weather year 2018 it is $[eqn]$ and compared to the most expensive single year, which is a lower (dual) bound on the objective, it is $[eqn]$ . Thus, MOD 2/Synthetic time-series leads to robust and on average cheaper solutions in ETHOS.FINE similar to the results in the gurobipy model.Figure 9. Feasibility testing of the energy system optimized for 1987 in 1994 before modification on the left and after modification on the rightThe cold dark lull period in early February (see Figure 5F) is marked in red. In the left graphic, due to insufficient backup capacity the supply gap variable has to be utilized meaning the energy system is not robust, i.e., there are still supply gaps after optimization.In the right graphic, the energy is fully operational during the cold dark lull after applying MOD 2/Synthetic time-series to the original optimization problem.

This also implies that optimization based on average or recommended reference years systematically underestimates the required cost for robust energy supply by $[eqn]$ in ETHOS.FINE.

Compared to the results of this work, Ryberg14 estimates a residual load of about 61 GW and additional backup capacity required of about 25 GW for Germany. The difference to the 118–125 GW found in this study can be attributed to the fact that in the integrated European setting that Ryberg14 used, dark lulls can be partly suppressed by electricity transmitted from regions not hit by that dark lull as well as differences in demand data.

In either case, the cold dark lull period is the most critical for CCGT—their installed capacity is mainly driven by a single dark lull period, as shown in Figure 9. Figure 9 also illustrates the effects of MOD 2/synthetic time-series. The left graphic shows the result of the ESM optimized for 1987 when testing its feasibility in 1994 reveals a supply gap. After applying MOD 2/synthetic time-series, the time period gets integrated into the optimization problem and after reoptimizing the supply can now be covered using existing capacities.

Modification MOD 3/Combine uses several types of cutting planes principles in one algorithm. It converges for all years, often only requiring one iteration of 2. Sometimes, multiple iterations of MOD 1/demand increase are necessary as well. The effect of MOD 3A/yearly balance is marginal: It does not effect model run times, nor results.

The total costs average out to 66.0bn€ per year, with a range of [61.2,72.1], which is slightly more than MOD 1/Demand increase. Thus, in terms of costs, the MOD 1/Demand increase is preferable. Figure 10 gives the results of optimizing each year independently for MOD 3/Combine. Notably, the results for each year are very similar to each other, suggesting robust solutions share some traits. Cheap solutions contain less onshore wind.Figure 10TAC comparison for robust solutions based on single years using MOD 3/Combine, a single node gurobipy model

Comparison of modifications

All three featured approaches generate robust solutions, i.e., solutions with no supply gaps, subject to numerical tolerances. The modifications’ performance is summarized in Table 4.

The results show a moderate increase in investment for robust models leading to TAC increases of $[eqn]$ compared to the best available lower (dual) bound on the cost of a robust solution. The dual bound is given by the maximum total cost for a single year, since the cost of a robust solution cannot be lower than the cost of a single year. Thus, the highest cost year is also the only one that could be robust (Clearly, this can be shown via a proof by contradiction, since each solution is assumed to be optimal and thus there must not be a feasible solution with a lower objective value). For gurobipy, the lower bound is 59.7bn€ annually. For the model in ETHOS.FINE, it is 113.6bn€ annually.

An upper (primal) bound is given by the maximum capacity for each technology per region and year, since this will be feasible for all years. This results in a range of [113.6,195.66]bn€ annually for an optimal robust energy system in ETHOS.FINE. For the gurobipy model, the range for optimal robust solutions is [59.7,67.6]bn€ annually.

Capacity changes in near-optimal robust solutions

It is important to acknowledge that any single robust solution may not accurately capture the properties of robust solutions as a whole, especially if considering the space of near-optimal solutions that may be relevant to decision makers, see the recommendations by Lombardi et al.30 While we did not explicitly perform modelling-to-generate-alternatives on the solution space, computing 40 extremal solutions on single years and iteratively robustifying them gives us a sample of 40 near-optimal robust solutions that allow us to infer some properties of near-optimal robust solutions:

Compared to optimizing individual years, near-optimal solutions to the robust model systematically use less onshore wind. This is plausible, since most years contain no extended (dark) lull periods coinciding with peak demands. In years without extended dark lulls, wind power provides stable and cheap energy, compared to PV that might require more storage and conversion units. However, in years with dark lulls this advantage disappears. As such, if costs for storage and conversion are priced in, optimizing a year without a dark lull may lead to more investment in onshore wind than would be efficient. Integrating appropriate dark lull periods, as suggested in this work, might help counteract that effect, leading to a more balanced energy mix. A functioning capacity market, especially for backup technologies, is essential to ensure the needed capacities are installed and ready to generate electricity during dark lull periods. Overall, robust solutions were only $[eqn]$ more expensive compared to the lower (dual) bound given by the most expensive single year. Contrarily, a model based on average or recommended reference years systematically underestimates costs by over $[eqn]$ .

Convergence

The gurobipy implementations of MOD 1/Demand increase and MOD 3/Combine tended to quickly converge. However, sometimes residual load differences in the order of MWh/kWh remained. In these cases, the algorithm was terminated after 20 iterations. MOD 2/Synthetic time-series needed between 1 and 54 iterations to converge in the gurobipy model. The number of iterations generally decreased for higher annual cost of individual years, but was mainly due to overwriting critical time periods where priority was given to critical time periods selected in earlier iteration steps. In ETHOS.FINE, this issue was prevented by evaluating critical time periods on the modified data largely preventing overwriting. The calculations then needed between 1 and 8 iterations for the 5 reference years. Here, one iteration incurs the same computational load of solving one ESM, or less if warm-starts help reducing computation times.

Full load hours and system cost

Lower investment in onshore wind capacity was a reoccurring pattern in the effects of modifications. The left graphic in Figure 11 shows the annual full load hours (FLH) for wind on- and offshore as well as PV compared to TAC for the respective models. For wind on- and offshore, these are strongly correlated (Pearson correlation coefficients of $[eqn]$ and $[eqn]$ for wind on- and offshore, respectively). The FLH of PV and the TAC are nearly uncorrelated (Pearson correlation coefficient of $[eqn]$ ). For wind, this mirrors earlier results of Gotske et al.,25 who showed similar correlations for an European System.Figure 11. The left graphic shows wind on- and offshore as well as PV full load hours compared to TAC in for all yearsThe right graphic shows combined cost for wind and for PV as share of the TAC compared to TAC for all yearsEach dot represents 1 year. Lines indicate regression lines of the different energy generator types.

The right graphic in Figure 11 shows the share of TAC of PV and wind, combined for on- and offshore, compared to the TAC. For wind, these are negatively correlated (Pearson correlation coefficient of $[eqn]$ ) while for PV, these are strongly positively correlated (Pearson correlation coefficient of 0.59) indicating that as full load hours of wind drop, wind capacity gets replaced with PV capacity. The two diagrams in Figure 11 together show that the TACs depend strongly on the availability of wind. In weather years with low full load hours for wind, an increase in PV capacities is observed indicating a higher reliance on PV in general which is also reflected in the robust years as investment increases for PV capacities for MOD 2/Synthetic time-series and MOD 3/Combine.

Application of results

The approaches outlined in this work can be used to make any ESM more robust to uncertain time-series data. As we showed, this serves to makes models more realistic and more suitable for real-world application preventing supply gaps for a moderate cost increase of 1.6%– $[eqn]$ .

Since uncertainties in demands or costs, which appear in the right-hand side (demands) or the objective (cost) of a mixed-integer linear program (MILP) can be reformulated as constraint-wise uncertainty, the methods outlined in this work can be applied to cost uncertainty as well. Another natural usage for the modifications proposed in this work is as part of a Benders decomposition framework. However, this was not the focus of this work, but the constraints that are added in each modification can equally well be used as feasibility/optimality cuts, as they serve to invalidate significant parts of the solution space. Here, the algorithmic performance for achieving robustness can be seen as a proxy for their potential value as Benders cuts.

Limitations of the study

The main drawback in this case study is that only one time-series for electricity demand in 2050 was used. The availability of future demand data is limited and is difficult to compare if taken from different sources. The selected data from Forschungsstelle für Energiewirtschaft e. V. (FfE)31^,^32^,^33^,^34 was chosen since it includes a severe cold period rarely observed in Germany. In combination with the weather years, the resulting operational conditions revealed several periods pairing low electricity generation with high demand as can be seen in Figure 5. Therefore, a high degree of robustness can be assured. Further demand time-series might still provide additional insights. This is especially the case if using weather years with an overall cold winter time and therefore a high amount of heating degree days, and thus varying total and local demand. The weather data used are based on historical weather years leaving out the effects of climate change, which very likely will influence the design of energy systems in the future and should be included in studies using these methodologies. The model used to develop the presented method is a Germany model without the possibility of imports or exports. While this island model approach was necessary for method development, the interconnectedness of the European energy system, as well as (cold) dark lulls covering large parts of Europe, is likely crucial, but has been neglected in this study. The Germany model proved sufficient for model development, but a more general model is necessary for more holistic recommendations. Assuming that years with disadvantageous distribution of sunny and wind hours as well as low full load hours are the exception, allowing the last time step to have a lower state of charge than the first would lower the conservatism of the system and reduce cost. This would require a measure of robustness that fully protects against a certain base uncertainty set, but allows using up some stored hydrogen for outlier events. One possible approach for that is outlined in Bärmann et al.35 At the same time, the model only provides an operational schedule under the assumption of perfect foresight within one year. In a more complex model setting, computing an operational schedule dynamically throughout the year might lead to some efficiency losses in the usage of energy, requiring additional investment to counteract this. Perfect foresight is assumed for the optimization of energy systems. This provides the advantage of being able to plan capacities with full information about cold dark lulls and overall energy generation. In a real-world setting, a higher degree of conservatism leading to additional generation or storage would be needed.

Summary of implications for planning energy systems

Our work contributes to the mounting evidence that suggest that using a fixed reference year and planning a renewable energy system based on that is unsuitable for practice.5^,^6^,^7^,^9^,^16^,^18 We show that not only does this lead to misallocation of resources, but to a systematic underestimation of costs for and investment in storage and conversion, i.e., Li-Ion battery capacity and CCGT. Thus, if energy systems are planned based on single years, policy makers need to separately assess how much, not whether, additional storage and conversion capacities are needed and be aware of the fact that this will incur additional costs. For practitioners who model energy systems, we propose three workable approaches that can ensure energy system designs are robust against a range of weather realizations. We also note that the total amount of wind hours per year is strongly correlated with energy system costs. Thus, for a conservative estimate, low-wind reference years are better suited.

Resource availability

Lead contact

Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Sebastian Kebrich (s.kebrich@fz-juelich.de).

Materials availability

This study did not generate new materials.

Data and code availability

The full code, model, and data are publicly available via GitHub.36 Any additional information required to recreate the results reported in this paper is available from the lead contact upon request.

Acknowledgments

This work is co-funded by the 10.13039/501100001659Deutsche Forschungsgemeinschaft (DFG, 10.13039/501100001659German Research Foundation)—2236/2. This work was partly supported by the 10.13039/501100009318Helmholtz Association as part of the Platform for the Design of a Robust Energy System and Raw Material Supply (RESUR) and the program, “Energy System Design.” This work was partly funded by the 10.13039/501100000780European Union (ERC, MATERIALIZE, 101076649). Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.

Author contributions

S.K., conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing – original draft, writing – review and editing, and visualization; F.E., conceptualization, methodology, software, validation, formal analysis, investigation, writing – original draft, writing – review and editing, and visualization; D.F., supervision, writing – original draft, and writing – review and editing; C.B., funding acquisition, resources, supervision, writing – review and editing; H.H., funding acquisition, resources, supervision, writing – original draft, and writing – review and editing; J.L., resources.

Declaration of interests

The authors declare no competing interests.