Day-Ahead Hourly Forecasting of Power Generation from Photovoltaic Plants

TL;DR

This paper compares simple and sophisticated forecasting methods for photovoltaic power generation across multiple plants and evaluates the influence of weather conditions and forecasts on prediction accuracy.

Contribution

It provides a comprehensive comparison of forecasting methodologies and assesses weather impact, offering insights into improving PV power prediction accuracy.

Findings

Sophisticated methods outperform simple ones in accuracy.

Weather forecasts significantly influence PV power prediction.

Methodology comparison across diverse PV plants enhances understanding.

Abstract

The ability to accurately forecast power generation from renewable sources is nowadays recognised as a fundamental skill to improve the operation of power systems. Despite the general interest of the power community in this topic, it is not always simple to compare different forecasting methodologies, and infer the impact of single components in providing accurate predictions. In this paper we extensively compare simple forecasting methodologies with more sophisticated ones over 32 photovoltaic plants of different size and technology over a whole year. Also, we try to evaluate the impact of weather conditions and weather forecasts on the prediction of PV power generation.

Figures

Tables

Table 1. TABLE I: Monthly error.

Error-Method / Month		Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec	Avg.
	GB	2.35	3.00	3.62	3.79	4.45	4.21	3.35	3.40	3.68	3.39	2.72	1.93	3.26
	NN	2.39	2.99	3.61	3.73	4.35	4.14	3.28	3.42	3.64	3.34	2.75	2.01	3.24
nMAE	kNN	2.32	3.00	3.55	3.64	4.30	4.06	3.08	3.28	3.53	3.26	2.58	1.73	3.13
(%)	QRF	2.29	2.95	3.57	3.58	4.35	4.09	2.97	3.26	3.58	3.22	2.55	1.72	3.11
	SVR	2.30	2.90	3.56	3.73	4.34	4.20	3.25	3.37	3.61	3.27	2.62	1.85	3.19
	ENS	2.25	2.90	3.49	3.54	4.23	3.99	3.02	3.22	3.49	3.20	2.54	1.73	3.07
	GB	5.97	7.07	7.68	7.82	9.05	8.35	7.92	8.43	7.91	7.37	6.64	4.91	7.36
	NN	5.94	7.00	7.65	8.01	9.19	8.44	7.97	8.62	7.92	7.30	6.64	5.18	7.41
nRMSE	kNN	5.80	6.86	7.45	7.72	8.94	8.27	7.74	8.55	7.66	7.11	6.49	4.85	7.21
(%)	QRF	5.93	6.93	7.73	7.80	9.15	8.43	7.66	8.57	7.83	7.22	6.59	4.89	7.32
	SVR	5.98	6.93	7.73	8.04	9.09	8.47	7.86	8.58	7.94	7.30	6.67	5.06	7.40
	ENS	5.76	6.78	7.46	7.67	8.88	8.19	7.65	8.45	7.66	7.08	6.45	4.78	7.16
	GB	102	128	136	140	154	148	114	129	142	133	115	84.8	126
	NN	104	128	134	135	149	146	115	130	140	132	116	87.7	125
MAE	kNN	102	128	134	132	146	140	108	125	137	127	108	73.4	120
(kW)	QRF	102	126	136	130	149	139	103	124	140	127	107	73.7	120
	SVR	101	124	134	135	148	146	116	128	140	130	110	79.1	123
	ENS	99.5	124	132	128	144	138	105	122	136	126	107	74.1	118
	GB	0.226	0.349	0.255	0.525	0.150	0.538	-0.0867	1.29	-0.396	0.527	0.545	0.398	0.313
	NN	0.431	0.422	0.385	1.31	0.675	0.658	0.154	1.60	-0.187	0.324	0.627	0.565	0.514
nMBE	kNN	0.487	0.429	0.367	1.03	0.545	0.313	-0.0841	1.29	-0.248	0.295	0.766	0.607	0.434
(%)	QRF	0.0912	-0.0406	-0.0874	0.751	0.313	0.296	-0.189	1.39	-0.342	-0.0519	0.377	0.238	0.156
	SVR	0.504	0.380	0.436	1.41	0.728	0.381	-0.0712	1.52	-0.279	0.255	0.697	0.720	0.493
	ENS	0.334	0.256	0.224	1.02	0.495	0.382	-0.0848	1.41	-0.287	0.191	0.589	0.487	0.356

Table 2. TABLE II: Average error (nMAE), plant by plant.

PV #	$P_{nom}$ [kW]	GB	NN	kNN	QRF	SVR	ENS
1	555	3.06	3.04	3.00	3.07	3.12	2.98
2	960	2.97	2.90	2.93	2.97	2.99	2.89
3	9950	4.23	4.17	3.98	3.96	4.13	3.93
4	9950	4.14	4.38	4.16	4.05	4.28	4.04
5	4170	1.92	1.78	1.76	1.73	1.77	1.72
6	3750	1.30	1.33	1.29	1.29	1.36	1.27
7	5250	3.02	3.10	2.97	2.93	2.98	2.91
8	3050	3.18	3.65	3.34	3.04	3.38	3.17
9	2260	2.89	2.84	2.79	2.77	2.85	2.72
10	1760	2.90	2.71	2.70	2.75	2.75	2.65
11	4991	4.00	3.82	3.58	3.57	3.65	3.56
12	1935	3.98	3.92	3.82	3.77	3.89	3.75
13	1070	3.36	3.32	3.25	3.21	3.33	3.16
14	2150	1.95	1.87	1.78	1.72	1.76	1.73
15	8723	3.71	3.66	3.57	3.71	3.61	3.55
16	1190	1.82	1.98	1.81	1.79	1.96	1.78
17	4834	3.57	3.63	3.40	3.35	3.48	3.30
18	6000	3.84	3.84	3.66	3.56	3.71	3.56
19	6600	3.90	3.87	3.74	3.73	3.86	3.67
20	4540	3.17	3.05	2.98	3.01	3.00	2.93
21	1560	3.94	3.85	3.64	3.65	3.75	3.63
22	9990	3.92	3.86	3.75	3.81	3.82	3.72
23	1410	4.20	4.25	4.10	4.06	4.07	4.04
24	1200	3.83	3.73	3.61	3.56	3.70	3.57
25	2630	2.94	3.02	2.94	2.95	2.98	2.89
26	2010	3.53	3.62	3.48	3.52	3.48	3.41
27	1000	3.78	3.84	3.70	3.63	3.72	3.61
28	8240	4.00	3.77	3.63	3.67	3.75	3.61
29	480	2.82	2.75	2.71	2.66	2.75	2.63
30	720	2.76	2.67	2.65	2.59	2.64	2.56
31	480	2.91	2.84	2.81	2.77	2.82	2.73
32	660	2.79	2.75	2.73	2.75	2.72	2.66

Table 3. TABLE III: Average nMAE error under different weather conditions.

CSI / Method	Number of hours	GB	NN	kNN	QRF	SVR	ENS
CSI $\le 0.1$	553	6.61	6.75	6.64	6.63	6.71	6.61
CSI $\le 0.2$	3600	12.5	13.3	13.8	13.1	13.0	13.1
CSI $\le 0.3$	9377	8.38	8.82	8.91	8.31	8.57	8.45
CSI $\le 0.4$	9311	6.90	7.27	6.95	6.51	7.01	6.69
CSI $\le 0.5$	7055	7.07	7.39	6.86	6.53	7.13	6.72
CSI $\le 0.6$	6200	6.77	6.98	6.42	6.25	6.84	6.37
CSI $\le 0.7$	7122	6.97	7.00	6.23	6.21	6.75	6.29
CSI $\le 0.8$	8366	6.63	6.54	5.94	6.08	6.46	6.00
CSI $\le 0.9$	13212	5.84	5.59	5.08	5.33	5.56	5.16
CSI $\le 1$	57324	6.38	6.11	6.06	6.16	6.07	5.94

Equations

$$nMAE=\frac{1}{N}\sum _{i=1}^N\frac{\left|\hat{P}(i)-P_m(i)\right|}{P_n(i)}\cdot 100,$$

$$nRMSE=\sqrt{\frac{1}{N}\sum _{i=1}^N{\left(\frac{\hat{P}(i)-P_m(i)}{P_n(i)}\right)}^2}\cdot 100.$$

$$nMBE=\frac{1}{N}\sum _{i=1}^N\left(\frac{\hat{P}(i)-P_m(i)}{P_n(i)}\right)\cdot 100,$$

$$MAE=\frac{1}{N}\sum _{i=1}^N\left|\hat{P}(i)-P_m(i)\right|,$$

$${\hat{P}}_{GB}(i)=c_1\cdot GTI(i)+c_2\cdot GTI(i{)}^2+c_3\cdot GTI(i)\cdot T(i),$$

$$w_i=e^{-\frac{d_i^2}{{\sigma }^2\cdot d_1^2}},$$

Full text

Keywords - PV plants, Machine Learning algorithms, power generation forecasts.

I Introduction

High penetration levels of Distributed Energy Resources (DERs), typically based on renewable generation, introduce several challenges in power system operation, due to the intrinsic intermittent and uncertain nature of such DERs. In this context, it is fundamental to develop the ability to accurately forecast energy production from renewable sources, like solar photovoltaic (PV), wind power and river hydro, to obtain short- and mid-term forecasts. Accurate forecasts provide a number of significant benefits, namely:

• •

**Dispatchability: **secure power systems’ daily operation mainly relies upon day-ahead dispatches of power plants [1]. Accordingly, meaningful day-ahead plans can be performed only if accurate day-ahead predictions of power generation from renewable sources, together with reliable predictions of the day-ahead load consumption forecasts (e.g., see [2]) are available;

• •

**Efficiency: **as output power fluctuations from intermittent sources may cause frequency and voltage fluctuations in the system (see [3]), some countries have introduced penalties for power generators that fail to accurately predict their power generation for the next day; thus, some energy producers prefer to underestimate their day-ahead power generation forecasts to avoid to incur in penalties in the next day. Such induced conservative behaviours are clearly not efficient;

• •

**Monitoring: **mismatches between power forecasts and the actually generated power may be also used by energy producers to monitor the plant operation, to evaluate the natural degradation of the efficiency of the plant due to the aging of some components (see [4]) or for early detection of incipient faults.

For the previous reasons, the topic of renewable energy forecasting has been also object of some recent textbooks like [5] and [6] that provide overviews of the state-of-the-art of the most recent technologies and applications of renewable energy forecasting. In this context, the objective of this paper is to compare different methodologies to predict day-ahead hourly power generation from PV power plants.

I-A State of the Art

Power generation from PV plants mostly depends on some meteorological variables like irradiance, temperature, humidity or cloud amount. For this reason, weather forecasts are a common input to forecasting methodologies for PV generation. Depending on the specific problem at hand, forecasts may be also necessary at different spatial and temporal scales, as from high temporal resolutions (i.e., of the order of minutes) and very localized (e.g., off-shore wind farms) to coarser temporal resolutions (e.g., hours) and covering an extended geographical area (e.g., a region or a country) for aggregated day-ahead power dispatching problems. At the same time, very different approaches and methodologies have been explored in the literature, based on statistical, mathematical, physical, machine learning or hybrid (i.e., a mix of the previous) approaches. For example, [3] uses fuzzy theory to predict insolation from data regarding humidity and cloud amount, and then uses Recurrent Neural Networks (RNNs) to forecast PV power generation. Autoregressive (ARX) methods are used in [7] for short-term forecasts (minute-ahead up to two hour-ahead predictions) using spatio-temporal solar irradiance forecast models. A forecasting model for solar irradiance for PV applications is also proposed in [8]. The presence of particulate matter in the atmosphere (denoted as Aerosol Index (AI)) is used in [9] to support an artificial neural network (ANN) to forecast PV power generation.

As for the specific day-ahead hourly forecasting PV power problem, [10] use add a least-square optimization of Numerical Weather Prediction (NWP) to a simple persistence model, to forecast solar power output for two PV plants in the American Southwest. A multilayer perceptron was used in [11] to predict the power output of a grid-connected 20-kW solar power plant in India. A stochastic ANN was adopted in combination with a deterministic Clear Sky Solar Radiation Model (CSRM) to predict the power output of four PV plants in Italy. A weather-based hybrid method was used in [13] as well, where a self-organizing map (SOM), a learning vector quantization (LVQ) network, a Support Vector Regression (SVR) method and a fuzzy inference approach were combined together to predict power generation for a single PV plant. In [14] Extreme Learning Machines (ELMs) are used to predict the power generation of a PV experiment system in Shanghai. Finally, we refer the interested readers to the two recent papers [15, 16], and to the references therein, for an extensive review of the literature.

A Global Energy Forecasting Competition (GEFCom2014) has recently allowed different algorithms to be compared, in a competitive way, to solve probabilistic energy forecasting problems, see [17] for a detailed description of the outcome of the competition. GEFCom2014 consisted of four tracks on load, price, wind and solar forecasting. In the last case, similarly to this paper, the objective was to predict solar power generation on a rolling basis for 24 hour ahead, for three solar power plants located in a certain region of Australia (the exact location of the solar power plants had not been disclosed to the participants of the competition) [17]. An interesting result of the competition was that all the approaches that eventually ranked at the first places of the competition were nonparametric, and actually consisted of a wise combination of different techniques.

I-B Contributions

The performance of the various forecast models are affected by many elements of uncertainties, and in the opinion of the authors it is not always clear how single choices (e.g., the choice of a specific prediction methodology over another) or different factors (e.g., meteorological forecasting errors) contribute to the final prediction error (i.e., in terms of predicted vs. actual power generation). In fact, most of the existing related papers, including the previous references, generally propose a single methodology to perform the power forecasting task, and compare their results with other very basic algorithms, while comparisons among different more sophisticated approaches can not be easily done. In fact, different authors have generally worked on different data-sets, and the final results can not be compared, as these depend on the specific period of the year where the forecasting error was computed (e.g., in winter time the error is usually lower given that the number of hours of non-zero power generation is lower); the error also depends on the specific country (e.g., it is simpler to predict a sunny day in summer in Italy than in other countries with a more irregular weather); and more in general the use of different error metrics, different weather forecasting tools with different accuracies, different sizes of PV plants, and different technologies or installations (e.g., roof-installed PVs vs. ground PVs) all also contribute to invalidate simple comparisons of different forecasting methodologies on the basis of the final accuracy results alone.

One exception to the previous consideration is provided by the already mentioned Global Energy Forecasting Competition 2014, where different (sophisticated) forecasting algorithms were in fact compared, and ranked, upon the same, publicly available, data-set. However, note that the competition only lasted less than three months, thus not allowing one to validate the final rank over different seasons, and only involved three PV plants. From this perspective, our work extends the results of the competition by further comparing the same algorithms that ranked at the first places of the competition over a longer horizon of time, and over a more variegate set of different PV plants.

Accordingly, more specifically, the contributions of this paper are the following:

• •

We extensively compare four state-of-the-art different “black-box” forecasting methodologies upon the same set of data (i.e., k-Nearest Neighbours; Neural Networks; Support Vector Regression; and Quantile Random Forest). Such methods have been tailored to address this specific task of interest. Note that k-Nearest Neighbours, Quantile Random Forests and Support Vector Regression were the building blocks of the algorithms ranked at the first places of the Global Energy Forecasting Competition for solar power forecasting (i.e., they ranked first, second and fifth respectively). We also added Neural Networks in the comparison as it has been frequently used by many other researchers, as from the previous state of the art;

• •

We further compare the predictions results with those obtained with a very simple second-order regressive method. While the previous more sophisticated methodologies usually outperform such a simple method, still the improvement is not as large as one might expect. Accordingly, second-order regressive methods may still be regarded accurate enough for most operations; this simple comparison is usually missing in the literature, thus not making it simple to evaluate the gain obtained by using more sophisticated methods;

• •

As a final term of comparison, we also consider an ensemble of all the previous methodologies that combines the forecasts obtained by the single algorithms. This combined method outperforms the single algorithms alone, thus confirming what has been observed in other examples as well (e.g., [18]), and consistently with the results in [17] as well;

• •

All results are obtained and validated upon real data recorded over 32 different PV plants of different sizes, technologies and characteristics in general, for an overall installed nominal power of about 114 MW, for an horizon of one year. Also, the availability of so many PV plants is also an element that is missing in most comparisons existing in the literature;

• •

In addition, we evaluate how the accuracy of the different algorithms vary under different weather conditions;

• •

Finally, we try to evaluate the component of the error that is solely due to the inaccuracy of the weather forecasts, by comparing the accuracy of the same methodology when the inputs are predicted variables from meteorological forecasts, and when the inputs are the actual measured variables.

This paper is organised as follows: Section II describes more in detail the case study, and other ingredients required to perform the comparison. Section III briefly describes the forecasting methodologies that we have used to predict power generation from PV plants. The obtained results are provided and discussed in Section IV. Finally, in Section V we conclude our paper and outline our current line of research in this topic.

II Case study

We here use data collected from 32 PV plants installed at different latitudes in Italy (i.e., Northern Italy, Central Italy, Southern Italy and Sicily). The size of the plants ranges from a few tens up to 10 MW, for an overall installed nominal power of about 114 MW. The technology used for the PV films includes monocrystalline silicon, polycrystalline silicon, thin-film amorphous silicon and flexible amorphous thin-film silicon. About half of the PV plants are installed on roofs, and half from the ground (we also had the knowledge of the tilt angles for all PV panels in the PV plants). The investigated set of PV plants was chosen to be a representative (scaled) set of the whole Italian PV installations (both in terms of technological mixture, and geographical positioning). The exact location of the PV plants is shown in Figure 1.

For this work, we have used data of the generated power from June 2014 up to end of 2016.

II-A Data

We can classify the available data as follows:

Meteorological data : We have used meteorological forecasts from two different providers; namely, Aeronautica Militare111http://www.meteoam.it/ and MeteoArena222https://www.meteoarena.com/. In particular, we had at our disposal the 24-hour ahead hourly forecast of direct, diffuse, and total irradiance on a horizontal surface on Earth; and the 24-hour ahead hourly forecast of temperature.

Measured data : We had at our disposal the actual hourly power generated from each PV plant. Also, we had the horizontal irradiance measured from satellite data (with a spatial resolution of 3.5 km $\times$ 3.5 km and a time resolution of an hour). We used satellite data to optimally combine the forecasting meteorological data from the two providers into a single value. While we found that such a combination would depend on the specific PV plant (as one meteorological forecasting provider may be more accurate than the other for a specific location), still we do not give here more details of such a combination procedure, as it is not the focus of this paper.

Computed data : We used the knowledge of the time of the day (and the year) to compute the sun altitude and azimuth. Then we used Perez Sky Diffuse Model (see [19]), together with the knowledge of the data of single PV plants, to infer the beam, the diffuse and the total irradiance upon the tilted plane of the panels.

II-B Training set and validation set

We used hourly data from 1 May 2014 until 8 November 2015 as our training set, and hourly data from 9 November 2015 to 12 November 2016 as a test set (5 weeks were actually not used due to the lack of information from one weather forecasting provider; this difference with respect to the other weeks of the year did not allow us to validate the results in those periods. Accordingly, there are 2 missing weeks in May 2016, and 3 missing weeks in August 2016). This implies that the test set is about one year long (48 weeks), which gives us the possibility to compare the accuracy of PV forecasts over all seasons. Also, every week we extend the training set to include the last available measurements, which is convenient both to increase the size of the training set, and also to take into account the latest effects (e.g., aging of the PV plants).

II-C Performance Indices

We use the normalised Mean Absolute Error (nMAE) as the main performance index to compare the different algorithms. In particular, nMAE is defined as

[TABLE]

where $\widehat{P}(i)$ is the predicted generated power at the $i$’th hour, and $P_{m}(i)$ and $P_{n}(i)$ are the actual measured power and the nominal power of that given PV plant respectively, at the same hour (note that the nominal power may actually change over time if part of the PV plant is unavailable, e.g., due to maintenance reasons). The prediction error is then averaged upon the number of hours $N$ of the comparison horizon. The nMAE is frequently used to evaluate forecasting errors, as it allows one to better compare the results obtained for plants of different size (see for instance [20]).

Other performance indices are however frequently used in the related literature as well. Among others, in this paper we shall further consider a normalised Root Mean Square Error (nRMSE), the Mean Absolute Error (MAE) and the normalised Mean Bias Error (nMBE) (see for instance [10] where the same performance indicators have been recommended as well). In particular, the nRMSE penalyses large errors more than the nMAE, and it is defined as

[TABLE]

Both the nRMSE and the nMAE do not retain the information on the sign of the error. For this reason, the nMBE may be useful, as it is defined as

[TABLE]

where a positive nMBE corresponds to an overestimation of the actual power generation. It is worth to mention that indices (2) and (3) were recently recommended by the European and International Energy Agency (IEA) for reporting irradiance model accuracy [21]. Finally, the MAE may be useful as, differently from all the previous indicators does not have normalisation factors at the denominator:

[TABLE]

III Methodologies

Many different methodologies may be adopted to predict power generation from PV plants. A typical classification of different methodologies relies on whether the physical equations of the PV plants are in fact used or not. For instance, reference [4] develops a novel electrothermal macro-model of PV power plants that takes as input variables the ambient temperature, irradiance and wind speed, and gives the generated power as an output. In the experience of the authors, such models are convenient in many situations (e.g., for initial plant design), but may not be effective for actual power generation forecast for many reasons: (i) such models are in any case (possibly accurate) approximations of the true plant, given the difficulty in modelling every possible physical phenomenon occurring in a complex real-world power plant; (ii) in any case, such models require a perfect real-time knowledge of all physical variables appearing in the power plant, which in most cases are not fully available; (iii) physical variables degrade over time anyway due to aging factors (so one should continuously monitor all variables all the time).

In order to overcome the previous shortcomings of physical models, many authors have adopted so-called “black-box” strategies, where the physical relationships between input variables and the generated power have been completely neglected, and let some Machine Learning (ML) methodologies (e.g., neural networks) choose the most convenient (arbitrary) relationship between input and output signals. Note that in this case one loses the physical meaning of the input-output relationship, and that the accuracy of such methods usually relies on the ability to have a large enough database that allows one to construct a reliable model. However, such methods are known to work rather well in practice (see for instance [3] and [9] for two specific examples, and [15, 16] for an overview of all ML algorithms employed in the literature for PV power forecasts). As a further example, all approaches that ranked highly in the GEFCom2014 solar competition were nonparametric [17].

Sometimes, reasonably accurate models can be achieved also if so-called “grey-box” identification methods are used. In this case, the input-output relationship is imposed from the outside, using some simplified physical relations (e.g., by neglecting high-order dependencies). As we shall see in the next section, such simple models may also provide accurate predictions.

The specific methodologies are now described in greater detail, while a schematic summary is provided in Figure 2.

III-A Grey-box model (GB)

We use here a very simple second-order model

[TABLE]

where $\widehat{P}_{GB}(i)$ is the predicted hourly generated power, using the Grey-box model, and $GTI(i)$ is the hourly forecast of the Global Irradiance, projected on the tilted plane of the panel. This simple model follows from the popular PVUSA model (see [22]) that is a simplification of the true relationship between temperature and irradiance as input variables, and generated power as an output. This model is widely used as a starting point for PV forecasting problems (see [23] for example). Parameters $c_{1}>0,c_{2}<0$ and $c_{3}<0$ are three model parameters that depend on the specific power plant (e.g., technology, area of the surface of the panels, angles), and are estimated from available historical data. In our specific applications, we found it to be more accurate by enforcing $c_{3}$ to be zero (i.e., neglecting the mixed term that both depends on the temperature and irradiance), and used a pseudo-inversion to obtain the values of $c_{1}$ and $c_{2}$ that best fitted the data. In our experience, we found it to be more convenient to update the parameters $c_{1}$ and $c_{2}$ every week, using only data of the last four weeks. The motivation for this is that, most likely, the optimal parameters of this simplified model change according to different climate conditions over the year, thus it is convenient to only consider the most recent history. Also, the conditions of the PV plants may change in time due to several factors (e.g., aging factors).

III-B Neural Network (NN)

We have used a static feed-forward neural network, with a hidden layer of 3 neurons, a sigmoidal activation function, and used a Bayesian regularization function for training. All choices were performed by trying different combinations on the same training set, and comparing the outcomes upon the validation set. The inputs were the same of the GB algorithm, and again, retraining was performed every week to possibly change the internal weights of the neural network. However, for simplicity, we decided not to change the structure of the neural network (e.g., number of neurons or activation function) during the test period.

III-C k-Nearest Neighbours (kNN)

The k-Nearest Neighbours is one of the simplest methods in machine learning. The rationale of this methodology is that a given weather forecast will most likely give rise to a power generation that will be very close to those in the past when there were similar weather conditions. Accordingly, one searches in the past data-base for $k$ weather forecasts that are the most similar to those of the hour of interest (i.e., the $k$ nearest neighbours in the name of the algorithm). Then, the corresponding historical power generations are combined (e.g., by giving more importance to those corresponding to the most similar weather conditions) to provide a single forecast of power generation. This algorithm has a number of free parameters (i.e., how to normalize the train set; how to compute the distance between the vector of predicted weather variables and a historical vector of the same variables; how many neighbours should be considered; and how to weigh the neighbours to combine the corresponding power generations to provide a single value). In this work, we have normalized variables so that each meteorological variable lies in the $[0,\ 1]$ interval; we have used Euclidean distance to compute the distance between two sets of meteorological variables; we have chosen $k=300$; and have finely used a weighted average to combine the outputs of the $k$ nearest neighbours, with weights chosen according to a Gaussian similarity kernel, i.e.,

[TABLE]

where the weight of the historical $i$th neighbour $w_{i}$ is computed as a function of its Euclidean distance $d_{i}$ from the weather forecast of the hour of interest. In (6), $d_{1}$ is the distance of the closest neighbour (i.e., $d_{1}=min\left\{d_{i}\right\},i=1,...,k$ and $\sigma$ is a parameter of the algorithm (in our case, we chose $\sigma=4$). The weights are then normalized to sum to unity. Note that this choice is slightly different from the conventional distance used by k-NN algorithms (that is the hyperbolic one), but is consistent with other similar works (see [2]). All parameters were learned over the training set and validated over the validation set.

In this case, we used a larger number of inputs, as in addition to the tilted global irradiance (GTI), we also used the diffuse (DTI) and the beam (BTI) components of the global irradiance, and the azimuth and elevation angles of the sun (the use of such inputs did not apparently give benefits in the case of the NN, but did actually give an improvement of the performance of the k-NN algorithm, in the validation set). In this case, we used the weekly “retraining” to increase the size of the historical dataset, but we did not change the parameters of the algorithm.

III-D Quantile random forest (QRF)

Random forest is a stochastic machine learning algorithm, originally developed in the ’90s and later extended by Breiman [24]. The random forest algorithm is actually an ensemble of classification models, where each model is a decision tree. The rationale behind this approach is that combining multiple classification models increases predictive performance than having a single decision tree. However, caution is required to build an ensemble of uncorrelated decision trees, and either boosting or bagging techniques can be used for this objective. In particular, random forest can be seen as a special case of bagging, where a further node splitting stage is added. In this work, we have used 300 trees, 5 (minimum) samples at the terminal nodes (i.e., leaves), Minimum Square Error (MSE) as objective function for the splitting method, and the hourly power as output variable. Finally, we have actually deployed the Quantile Random Forest variant of the algorithm, see [25], that, differently from conventional random forests, takes track of all the target samples, and not just of their average. As a further parameter at this regard, we have obtained 0.4 as the optimal value of the quantile parameter. All values have been obtained upon (again) the same validation set.

Again, in the case of the random forest, we have found it convenient to use the same number of inputs of the k-NN. While we have maintained again the same structure of the algorithm (i.e., in terms of number of trees and quantile parameter) during the weekly retraining, yet note that, at least in principle, the retraining causes the determination of a different decision tree every week.

III-E Support vector regression (SVR)

Support vector regression methods were originally proposed in [26, 27] as an application of Support Vector Machine (SVM) theory to regression estimation problems. One of the main characteristics of SVM techniques is that the so-called structural risk is minimised in the training phase, rather than the output training error (e.g., as in NNs). Among the many existing formulations of SVR, here we used $\nu$-SVR [28], where the parameter $\nu\in(0,1]$ represents an upper bound on the training error and a lower bound on the fraction of used support vectors. In addition to $\nu$, the algorithm requires also a meta-parameter $\gamma$, that is required to define the Gaussian kernel function, and $C$ that is the regularization parameter.

Again, for SVR we used the same inputs of the k-NN and QRF, and the same validation set, where we obtained $\nu=0.5$, $\gamma=1.25$ and $C=1$ as the optimal values of the parameters, and did not change them during our weekly retraining.

III-F Ensemble of methods (ENS)

The last method included in the comparison is the ensemble of all of the previous methods. The main idea behind the ensemble methodology is to weigh several individual independent forecasting techniques, and combine them in order to obtain another forecast that outperforms every one of them [18]. In fact, as noticed among others in [18] again, human beings tend to seek several opinions before making any important decision.

While different ways can be devised to combined single forecasts, here we have used the so-called stacked generalization [29], that simply computes a weighted average of the outputs of the previous algorithms. The optimal weights were here simply computed by minimising the square error in the validation set between the powers predicted by every single method, and the actual generated powers (i.e., using the Moore–Penrose pseudoinverse). Then we normalised the weights to have a unit sum.

While this was a simple way to combine the different forecasts, still it eventually outperformed all the other techniques singularly adopted, as it will be described in the next section.

IV Experimental results

Figure 3 compares the generated (measured) power versus the predicted powers, obtained with the six compared forecasting methodologies, for a PV plant of nominal size of 4834 kW (i.e., PV plant number 17 in Table II). The comparison was performed in seven following days in April 2016. Both the time range and the specific plant were here randomly chosen just for the purpose of this figure. In particular, the figure shows the typical characteristic bell-shaped curves of solar power generation, and the generated power is practically zero at night time.

More detailed results of the comparison are given in Figure 4, that shows the weekly nMAE error (computed according to Equation (1), and aggregated and averaged over an interval of a week for all 32 plants) obtained by each forecasting methodology.

Two main interesting aspects can be noted from Figure 4. The first one is that the errors depend on the particular season of the year. In particular, errors are larger in May and September, when solar power generation in Italy is still relatively high, but the weather is more unpredictable than in summer months. A second observation is also a very simple model like the GB is generally accurate at predicting the correct power generation shape. Indeed, its prediction rather overlaps those of the other methodologies. This shows that a second-order approximation of the true physics underlying the solar power generation problem may already provide an accurate forecast, if the method is tuned in a careful way. Still, other more complex methodologies do outperform GB, and are more convenient when a better performance is required.

A more detailed information regarding the comparison, together with other evaluation metrics, are shown in Table I. From Table I it is possible to appreciate that the combination of single methodologies (ENS) outperforms the single techniques. Also, ENS, QRF and kNN usually provide the three best performance, while SVR, NN and GB provide in general less accurate forecasts. This result is aligned with that of the GEFCom2014 competition ([17]) where QRF and kNN had outperformed all other methods (including SVR). Also, the fact that ENS is the best performing algorithm confirms the theoretical results of [18] and is also consistent with the experimental results of other works, see for instance [30, 31], indicating that the combination of several different forecasting tools is the best approach for the prediction of solar power generation.

The index nMBE deserves a special discussion: this index evaluates the signed bias of the algorithms. Thus, a small value of nMBE does not imply an accurate forecast (but rather, that errors compensate each other). While all algorithms have a small value of the nMBE, still it is possible to see that all the algorithms eventually overestimate power generation (between $0.156\%$ and $0.514\%$). Also note that ENS does not provide the best performance in terms of nMBE as it provides an intermediate forecast that tends to mitigate the error dispersion, but not its error bias.

Similarly, Table II compares the six forecasting algorithms over every single PV plant, where the nominal sizes of the plants are reported as well. The results in Table II confirm that ENS consistently outperforms the other algorithms (with the three plants being an exception). This further confirms that despite the differences are not huge, still they are consistently true.

The full normalized histogram of the (average) daily nMBE error is also shown in Figure 5 for each one of the six methods, together with the corresponding (truncated) kernel-smoothing estimation of the probability density function (PDF) (see [32]). From this figure, it can be also noticed that there are some days when the nMBE is very large (bigger than 20%). We also noticed that large occurrences of the nMBE happen at the same time for all methods. This could thus be caused by a completely wrong weather forecast or, more likely, to a wrong measurement of generated power, or to other unknown failures (or not communicated maintenance works) that occurred in the monitored PV plants.

Finally, from the histograms it is possible to appreciate again that in our analysis ENS outperforms the other algorithms. In fact, the lowest three values of the variance of the PDFs were obtained by the ENS, QRF and kNN methods respectively (ENS:$9.93$, QRF:$10.02$, kNN:$10.13$). Given the (relatively) small improvement, we validated this result using the signed rank Wilcoxon test [33], considering a significance level of 5%. The test can be used to compare two algorithms at the same time, and we used it to assess the statistical significance of the nMAE difference, by using all the hourly error values. As a result, after comparing every possible pair of forecasting algorithms, we obtained that all the differences could be believed as statistically significant, with the highest value (i.e., smallest difference) obtained when comparing kNN and QRF (p-value equal to 0.019). Even in such case however the values are still well below the 5% threshold, and the differences may still be regarded as significant.

IV-A Dependence on weather conditions

It is well known that weather conditions have an impact on power generation from PV plants, as well as on the accuracy of its prediction (see for instance [13]). In this section we shall both evaluate how the accuracy of the predictions changes under different weather conditions, as well as what algorithms are most performing in such circumstances. For this purpose, we shall use the clear sky index (CSI) to distinguish different weather conditions. The clear sky index is defined as the ratio between the actual global insolation measured at the site, and the global insolation expected if the sky were clear [34]. Many authors have used the CSI for this purpose, and for instance the authors of [35] have noticed that there may be a considerable overestimation of the irradiance for cloudy situations with CSI index between 0.3 and 0.8, while the actual measured irradiance may be underestimated when the CSI is lower than 0.2.

Table III reports the average hourly nMAE error obtained in all PV plants by each method for different values of the CSI.

As can be seen in Table III, predictions are less accurate for values of the CSI between 0.1 and 0.2, while they are more accurate for values greater than 0.8. This result is consistent with what had been observed in [35]. In comparative terms, it is also possible to see that the GB method is the most performing one when the sky is cloudy, QRF provides the best performance for intermediate values of the CSI, while the kNN is the best methodology when the sky is clear. Overall, the GB is clearly penalised by the fact that in Italy high values of CSI occur most frequently (see the second column of Table III). In this framework, the ensemble is still the most valuable approach as it always provides one of the best forecasts under any weather condition.

**Remark: **note that the CSI is not well defined for night hours, when the expected insolation at the denominator is zero. Accordingly, night hours were not considered in Table III, and this gave rise to larger nMAE errors than in the previous comparisons and tables.

IV-B Importance of accurate weather forecasts

The accuracy of the prediction of the expected energy produced by PV plants heavily depends, among others, on the accuracy of available meteorological forecasts. Here, we are interested in evaluating the margin of improvement that can be gained by having more accurate (possibly exact) meteorological forecasts. In particular, we now use the available irradiance measured from satellite data (with resolution of 3.5 km $\times$ 3.5 km) as an alternative to the GTI forecasts as an input to the GB method. The satellite data that we had at our disposal had a time resolution of 1 hour (and provided data aggregated with a sampling time of 15 minutes). Results are illustrated in Figure 6 and show that an improvement of about 1% can be consistently obtained over the whole year.

Although satellite data are clearly affected by some measurement errors (e.g., due to the resolution of the data), still they show that a significant improvement can be obtained with respect to the use of meteorological forecasts.

**Remark: **we noticed that similar improvements are obtained with the other methods as well, by substituting the predicted GTI with the measured one (with the ultimate average error of ENS around 2.00%). However, we have only reported the results for the GB, as in this case we could directly substitute the predicted GTI with the measured one, and we could directly evaluate the impact of (not perfect) weather forecasts. On the other hand, in the case of QRF for instance, we did not have the measurements of some input variables (e.g., BTI or DTI) to evaluate the same impact.

V Conclusion

Despite the rich literature on forecasting techniques for PV plants, the lack of thorough comparisons among different techniques in (spatially and temporally) extended datasets motivated the present work. Among the main results of this paper, we have evaluated the difference in the accuracy of simple methodologies (GB method) and more complicated approaches (QRF method, and ensemble of methods). While we noticed that the difference is not too great (an overall improvement in the nMAE of about 5%), still the improvement was shown to be consistent, and statistically relevant, over all PV plants and months of a year. Different methodologies appear to be the most suitable under different weather conditions (GB providing the best results in cloudy conditions, QRF for intermediate values of the CSI, and the kNN in sunny conditions). In this context, the ensemble has the advantage of providing one of the best forecasts under any weather condition. We also showed that more accurate weather forecasts of the irradiance alone would already improve the accuracy from (about) 3% to 2%.

In our experience, the residual $2\%$ error is mainly due to the fact that measured data (and forecasts as well) had a time resolution of 1 hour. Accordingly, they may fail to notice some effects occurring at a much faster time scale. Other sources of errors are that the irradiance (and its beam and diffuse components) are not obviously exactly the same everywhere in the PV plant; a PV plant contains a number of mechanical/electrical elements that in real life may give rise to some unexpected behaviours; and, finally, we are aware that there may have been some measurement errors, or some approximations, or even some transcription/registration errors in the available datasets.

As a final remark, in our experience we noticed that all methodologies performed in a better way if temperature data were not considered. Given that it is well-known that temperature is a variable that does contribute (though through higher-order terms) to the actual energy generation, our intuition is that the predicted temperatures that we had at our disposal were not accurate enough to in fact improve our forecasts. To validate this hypothesis we have started collecting temperature measurements to evaluate the ultimate improvement of having this further variable, at least in the ideal case where meteorological variables are measured. This will allow us to assess the sensitivity of each method to this specific input parameter.