Some examples of application for predicting of compressive sensing method

TL;DR

This paper evaluates the SALSA algorithm for forecasting electrical signals, temperature, and stock prices, demonstrating its superiority over linear extrapolation and comparing it to causal smoothing extrapolation.

Contribution

It applies SALSA to diverse real-world data and compares its forecasting performance with existing methods, highlighting its strengths and limitations.

Findings

SALSA outperforms linear extrapolation in all tested cases.

SALSA provides more accurate range predictions for electrical signals.

Causal smoothing extrapolation is more conservative for complex systems.

Abstract

This paper considers application of the SALSA algorithm as a method of forecasting and applies it to simulated electrical signal, temperature recording from the Australian Bureau of Meteorology and stock prices from the Australian stock exchange. It compares it to basic linear extrapolation and casual smoothing extrapolation, in all cases SALSA extrapolation proves to be a better method of forecasting than linear extrapolation. However, it cannot be imperially stated that it is superior to Causal smoothing extrapolation in complex systems as it has a higher L2 euclidean in these experiments. while usually retaining more shape and statistical elements of the original function than Causal smoothing extrapolation. Leading to the conclusion the Causal Smoothing extrapolation can provide a more conservative forecast for complex systems while the SALSA algorithm more accurately predicts the range of possible events as well as being the superior forecasting method for electrical signals, the physical process it is designed to forecast.

Tables

Table 1. Table 1: Table of values of mean residual per point for values of μ 𝜇 {\mu}

$\mu$	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0
Residual per point	2.0767	2.1702	2.1743	2.1936	2.1788	2.1135	2.2526	2.1391	2.1597	2.1237
$\mu$	1.1	1.2	1.3	1.4	1.5	1.6	1.7	1.8	1.9	2.0
Residual per point	2.2009	2.1778	2.2461	2.2261	2.1243	2.1720	2.1888	2.1460	2.1852	2.2436

Table 2. Table 2: Statistical Results of Electrical signal 0.2 second forecast

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	-2.82	-2.814	-2.736	Total L2 residual	41.044	11.5482	19.8732
Max	2.684	2.682	2.558
Mean	-0.00531	3.08e-05	0.00112	Total L2 residual per point	0.0456	0.0128	0.0221
STD	1.468	1.467	1.404
Range	5.504	5.496	5.294

Table 3. Table 3: Statistical Results of Electrical signal 1.0 second forecast

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	-2.82	-2.777	-2.658	Total L2 residual	306.2743	61.9608	258.2352
Max	2.684	2.672	2.542
Mean	-0.00531	-0.01101	-0.00812	Total L2 residual per point	0.3437	0.0695	0.2898
STD	1.475	1.466	1.336
Range	5.504	5.449	5.2

Table 4. Table 4: Statistical Results of BOM 2 Day Maximum temperature forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	14.2	15.6	13.66	Total L2 residual	2.1800e03	4.8381e03	712105e03
Max	37.7	33.6	33.32
Mean	23.05	22.95	21.66	Total L2 residual per point	8.5492	18.9730	27.9256
STD	5.054	4.320	3.942
Range	23.5	18	19.65

Table 5. Table 5: Statistical Results of BOM 2 Day Minimum temperature forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	1.7	5.38	3.915	Total L2 residual	1.8570e03	3.7586e03	6.1947e03
Max	21.8	18.31	17.21
Mean	11.43	11.37	10.18	Total L2 residual per point	7.2822	14.7395	24.2931
STD	3.902	3.12	2.766
Range	20.1	12.93	13.3

Table 6. Table 6: Statistical Results of BOM 7 Day Maximum temperature forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	14.2	16.11	13.53	Total L2 residual	3.3970e03	6.5986e03	8.13195e03
Max	37.7	32.83	33.87
Mean	23.25	23.31	21.67	Total L2 residual per point	12.7229	24.7137	30.4564
STD	5.104	4.482	4.352
Range	23.5	16.71	20.35

Table 7. Table 7: Statistical Results of BOM 7 Day Minimum temperature forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	1.7	5.685	3.725	Total L2 residual	2.7701e03	5.0055e03	6.3337e03
Max	21.8	17.6	17.28
Mean	11.63	11.56	9.845	Total L2 residual per point	10.3750	18.7474	23.7216
STD	3.937	3.14	2.504
Range	20.1	11.91	13.56

Table 8. Table 8: Statistical Results of ASX 2 Day Maximum price forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	67.92	68.34	68.8	Total L2 residual	150.9761	194.0201	5.8375e03
Max	77.66	76.86	76.82
Mean	73.12	72.98	71.86	Total L2 residual per point	0.9320	1.2437	36.034
STD	70.25	2.13	72.88
Range	2.242	8.52	8.02

Table 9. Table 9: Statistical Results of ASX 2 Day Minimum price forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	-67	67.56	67.24	Total L2 residual	150.3173	195.6079	5.7525e03
Max	77.17	75.99	76.32
Mean	72.18	72.05	70.97	Total L2 residual per point	0.9279	1.2539	35.5091
STD	2.237	2.119	8.179
Range	10.17	8.437	9.08

Table 10. Table 10: Statistical Results of ASX 5 Day Maximum price forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	67.92	68.83	68.8	Total L2 residual	296.9839	354.9674	5.8375e03
Max	77.66	77.64	76.74
Mean	73.12	73.2	71.66	Total L2 residual per point	1.8332	2.1912	36.034
STD	2.242	2.154	8.25
Range	9.74	8.807	7.94

Table 11. Table 11: Statistical Results of ASX 5 Day Minimum price forecasts

Data Type	Raw Data	Casual Extrapolation	Salsa Extrapolation	Comparison method	Casual Forecast	Salsa Forecast	Linear Forecast
Min	67	68.02	67.9	Total L2 residual	283.3359	315.1025	5.7525e03
Max	77.17	76.89	76.03
Mean	72.18	72.26	70.89	Total L2 residual per point	1.749	1.9451	35.5091
STD	2.237	2.167	8.163
Range	10.17	8.871	8.13

Equations

$$(Q^{\ast }z{)}_k=\frac{\Omega }{\pi }\sum _{t=q}^{s}sinc(k\pi +\Omega t)z(t)$$

$$R_{km}=(\frac{\Omega }{\pi }{)}^2\sum _{t=q}^{s}sinc(m\pi +\Omega t)sinc(k\pi +\Omega t)$$

$$R_v=R+vI$$

$$y_k=R_v^{-1}{Q}^{\ast }x$$

$$\hat{x}(t)=Q{y}_k$$

$$\hat{x}(t)=(Qy)(t)\frac{\Omega }{\pi }\sum _{t=q}^{s}sinc(k\pi +\Omega t)$$

$$\underset{x,vϵR^n}{\min }=\frac{1}{2}||Ax-y|{|}_2^2+\omega \sigma (v)$$

$$y=Ax$$

$$y=\begin{tabular}[]{|c|}y(0)\\ y(1)\\ ...\\ y(M-1)\end{tabular}\ ,\ x=\begin{tabular}[]{|c|}x(0)\\ x(1)\\ ...\\ x(N-1)\end{tabular}$$

$$||x|{|}_2^2=\sum _{n=0}^{N-1}|x(n){|}^2.$$

$$Ky=Ax$$

$$\hat{x}(t)=\frac{(z(t_0)-z(t_0)-A)}{A}(t)+(z(t_0)-\frac{(z(t_0)-z(t_0-A)}{A})t_0$$

$$z(t)=A(t)z(t-1)+\gamma (t),t=-1,1,2,.....$$

Full text

††School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, GPO Box U1987, Perth, 6845 Western Australia.

1 Introduction

Forecasting future events has always been both an extremely worthwhile and difficult endeavour, and while many processes have been able to predicted by improving our fundamental understanding of the underlying process, not all processes are simple and stable enough to mathematically predicted. The purpose of this project is to do a comparative study on the accuracy of extrapolations with the algorithm known as “SALSA” and a previously explored method “Causal Smoothing Extrapolation”. Applying both to real world data and evaluating how accurately each of algorithms can predict data in comparison to each other and evaluating how useful these forecasts may be. Causal Smoothing Extrapolation proposed in “On Causal Extrapolation of sequences with application to forecasting” [1] was the primary focus of the previous work [2], in which it was determined that in most cases more Causal Smoothing Extrapolation creates more accurate forecasts than linear extrapolation. This was explored using financial and regional maximum temperature time series data. The following next step in the investigation of Causal Extrapolation would be to compare it with a more complex extrapolation method. “Split augmented Lagrangian Shrinkage algorithm” or “Salsa” first proposed in “ Fast Image Recovery using Variable splitting and Constrained Optimization” [3] is a method of data extrapolation currently under study by todays mathematics community and is there for the most relevant method to compare to Causal Smoothing Extrapolation.

2 Theory

2.1 Causal Extrapolation Summary

The method is based on approach from [1]. The following is an extract from “Applications of band-limited extrapolation to forecasting of weather and financial time series” [2] which provides a summary of how the Causal Smoothing Extrapolation works.

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

This shows the progression of time series data z(t) for $s\leq t\leq q$ from raw data to Causally Smoothed $x(t)$ points by operators $Q^{*}$, $Rv$ and $Q$. Where$\omega$ and $v$ are constants selected by the simulation of 111 data points repeated 10,000 times in order to select the constants which give the most accurate projections. This topic is fleshed out further in “Applications of band-limited extrapolation to forecasting of weather and financial time series” [2] while the full proofs and workings of this algorithm can be found in [1].

2.2 Split Augment Lagrangian Shrinkage Algorithm or SALSA

SALSA is a method of image restoration and reconstruction where the goal is to minimise the following function by spliting the two major components of the functions and minimising them independently.

[TABLE]

Here component one $f(x)=\frac{1}{2}||Ax-y||_{2}^{2}$ and component two $f(x)={\omega}{\sigma}(v)$. The algorithm for forecasting a signal is done by comparing a signal y with length M to a bases vector x length N transformed by A which is a MxN matrix and minimise the difference between the two.

[TABLE]

[TABLE]

With multiple repetition of the SALSA algorithm the bases vector x and transform A can completely replicate the original signal y, when values of y are unknown it can interpolate and extrapolate the missing values of the original signal.

The complete algorithm is the following:

1. Set k=0,choose${\mu}>0,\ v_{0}\ and\ d_{0}$.

2. Repeat point

3. $x_{k+1}=argmin_{x}||Ax-y||_{2}^{2}+{\mu}||x-v_{k}-d_{k}||_{2}^{2}$

4. $v_{k+1}=argmin_{v}{\omega}{\sigma}(v)+\frac{\mu}{2}||x_{k+1}-v_{k}-d_{k}||_{2}^{2}$

5. $d_{k+1}=d_{k}-(x_{k+1}-v_{k+1})$

6$.k{\leftarrow}k+1$

7. Continue until stopping criterion is satisfied.

Here $||x||_{2}^{2}$ is defined by the $L_{2}$ norm

[TABLE]

The algorithm will initialise with zero vectors $d$ and $v$ of the same size as $x$. The parameter ${\mu}$ is the penalty parameter which will be determined by simulation, ${\sigma}$ is the regulation parameter(P in the appendices code) which can be found mathematically as N the length of the basis vector and ${\omega}$(lambda in the appendices code) is the regularizer also known as amount the white noise found in the signal. Elements in the basis vector x are updated with each iteration for as many repetitions as required. This can be summarised as the following, after choosing initial conditions ${\mu}$¿0, $v_{0}$, and $d_{0}$ apply to the time series bases vector $x_{k}$ to get $x_{k+1}$ repeat until the values of the bases vector set can be transformed into a vector which closely resembles the original data set. For my purposes I have chosen to use the “Fast Fourier transform” in Matlab to convert the original signal y to basis set x and the “Inverse Fast Fourier transform” to convert it back. Circumventing the need to directly define the transform A. I have also used the function ‘soft’ from Ivan Selesnicks SALSA toolbox in order to calculate the argmin values needed in step 3 and 4 of the algorithm [4]. Finally, in order to forecast using SALSA part of the original signal is masked using matrix K. K is and MxM identity matrix with values missing to represent a patchy or incomplete signal, since I am only using SALSA for forecasting the matrix K will obscure the last 2-10 values of y. The vector Ky will be used in place y in the algorithm.

[TABLE]

For further information on this please look into the lecture notes provided by Ivan Selesnick [4].

2.3 Linear Extrapolation

Linear extrapolation has also been included in portions of this paper in order to provide a base line for how the previous two methods compare to a standard trend line. The following equation is used in all linear forecasts in this paper.

Linear extrapolation for A historical Data points:

[TABLE]

3 Monte Carlo Simulation for SALSA forecast

Since d,v,${\sigma}$ have been defined previously this leaves values ${\mu},{\omega}$ and the length of the bases vector N to be selected. In order to do this a Monte Carlo simulation was created to test the most effective values for these constants. This follows the same process used in the previous experiments with Casual extrapolation [1,2]. The Monte Carlo simulation is structured as follows, A(t) takes random values within a uniform distribution, ${\omega}(t)$is standard gaussian white noise and ${\rho}$ is simply 1 [1]. This is to ensure that the process appears to be entirely random and cannot be forecasted by some other method.

[TABLE]

Here A(t) takes random values within a uniform distribution, $\gamma(t)$ is the standard Gaussian white noise.

We used $0.1\leq\mu\leq 200$ in steps of 0.1, $1\leq\omega\leq 10$ with integer steps and $100\leq N\leq 1000$ in steps of 100 with 10,000 trials each forecasting 7 points for a total of 70,000 total data point per trial was the original testing circumstances in order to provide a complete overview and find the most effective values, however this test was abandon after 72 hours of simulation when no significant reduction in the L2-Norm was noticed was noticed past ${\mu}$ of 0.6 , ${\omega}$ of 1 and N of 200. This has henceforth been used at the as the values for future forecasting.

4 Electrical signal experiements

The electrical signal information is a simulated data set provided by Dr. Nikolai Dokuchaev to the author.

It consists of 1000 signals lasting for 3 minutes each. Two experiments were done using this data set, the first was a 0.2 second forecast repeated 450 times and 1 second forecast repeated 90 times. Both forecasting methods were allowed 91 points of data in order to predict the next 2-10 points. The Results of which are below, linear extrapolation has been intentionally omitted from these graphs in order to improve the clarity of the following figures.

In both cases SALSA extrapolation has far lower L2 euclidean per point then linear extrapolation and casual smoothing extrapolation showing it to be far better extrapolation method for this process. This makes sense as the SALSA algorithim has been designed with the goals of processing and recovering signal data, it maintains a L2-euclidean of 0.0456 and 0.0695 per point for the 0.2 second and 1.0 second forecasts respectivly. This is 58% of its nearest competitors residual in the 0.2 forecast and 24% in the 1.0 second forecast. It also does not heavily compress the range of data points maintaining 94-96% of the range of the raw data set . The inaccuracy of Casual smoothing in this case is highly visable on the 1 second forecast data, in which forecasts maintains a flat trajectory while the data sharply assends or decends. This highlights the need to forecast using methods that relate to the physical process whenever it is avalible and save Casual smoothing extrapolation for when the process is completely unknown.

5 Bureau of Meteorology Experiements

Four experiments have been conducted using information from the Australian Bureau of Meteorology, maximum temperature forecasts of the next week made with 91 points of data repeated 38 times during the year, forecasts for the next 2 day with the same data repeated 133 times during the year and both tests repeated for minimum temperature forecasts. Data from the Australian Bureau of meteorology was retrieved from the “Perth Metro” station located -31.9192 Latitude ,115.8728 Longitude at an altitude of 25.9 meters [4] [5]. Data ranged from the Stations first open in January first 1994 to the twenty fifth of October 2018. This data was selected as it had an estimated 100 completeness for Maximum air temperature data and over 2 decades of collected data as well as being the local weather station. For the purposes of testing the complete year 2017 was used.

In these experiments the superiority of Casual smoothing extrapolation is clear with the L2- euclidean in the temperature data being 45-55% smaller then its nearest competitor.Though while its predictions are far closer to the actual points then the other two methods it compresses the data range to 65-77% of the raw data range. Comparitivly the Salsa extrapolation compresses the data to 67-84% of the raw range. While both perform significantly better linear extrapolation and neither have a forecast so poor that the maximum temperature forecast is below the minimum temperature for the day of vice versa for both experiements. Casual smoothing extrapolation appears to be a significantly more useful forecasting tool for this particular data set.

6 Australian stock exchange experiements

Similar to the previous section 4 tests were conducted using data from the Australian stock exchange, with 2 day and 5-day forecasts of minimum and maximum prices repeated 79 and 31 respectfully with moving 91-point data sets. The data set for the testing of stock prices were obtained from Market index [6]consisting of the opening, high, low and close values of a tradeable stock and index fund between the 11th of October 2017 and the 10th of October 2018. These values were also confirmed with Commonwealth Securities Limited quotes [7] of these data points.

This series of experiments had similar results to the temperature forecasts with Causal extrapolation being the superior forecasters with the most compressed data set. The main difference being that in three out of the four cases the Causal extrapolation had ranges closer to raw data range than the SALSA extrapolation, forecasting data which visually resembles the shape of the underlying process more. The SALSA extrapolated points also had forecasted maximum prices which fell below the minimum price forecasted and minimum forecasted prices which were above the maximum price forecasted, while Casual smoothing extrapolation did not. There for if SALSA extrapolation was used to forecast prices in an attempt to purchase and sell stocks during a 2- or 5-day periods you would have a maximum price falling lower than its minimum price resulting in a catastrophic loss if trades were made with an autonomous system. Causal smoothing extrapolation does not have as large a chance of experiencing this catastrophic failure because at no point does its forecast for the maximum price fall lower than the forecast for the minimum price or vice versa. For this data type the more conservative and reliable forecasts made by casual smoothing extrapolation is clearly the better choice.

7 Summary and Conclusion

In all cases the primary methods of forecasting discussed in this paper are superior to a basic linear forecast when the underlying processes are unknown and Salsa extrapolation is clearly superior in the known process. In the case of the complex underlying process Casual Smoothing extrapolation had a lower L2 euchlideon then Salsa extrapolation, resulting in a forecast more likely to be closer to the actual value that will occur. This is ideal for the Australian Stock exchange data as it will result in the minimum amount of loss when conducting trades and making trades closer to the minimum and maximum prices on any given day. It also has not produced a forecast in which a loss would occur if a stock was sold at the maximum forecast price and bought at the minimum forecast price, while this has occurred multiple times with SALSA extrapolation. This is not to say that either processes would be guaranteed to make a profit on any give day or over time, but the Salsa extrapolation has the greatest capacity to incur a long-term loss or catastrophic short-term loss for an autonomous trading process. In the case of the weather data which is the superior method of prediction is not as straight forward as no cross over of the data between minimum and maximum forecasts occurs. Though the casual smoothing extrapolation had a L2 euclidean half of the SALSA forecasts and a range at most 82% of the SALSA range. This would result in the Casual smoothing forecasts having far more accurate forecasts of the expected maximum and minimum temperatures for any given day while the SALSA forecast would produce a winder band of temperatures which could prove more useful for preparing for what kind of weather tomorrow may bring. The forecasting of electrical signals is the most simple series to discuss as the SALSA forecasts are more accurate, barely compressed and are completed faster than Casual smoothing extrapolation. This makes Casual smoothing extrapolation a redundent method for forecasting electrical signals as forecast period was shorter then the time it took to complete a prediction. SALSA extrapolation on the other hand is shown to be a forecasting processes that is highly practical for making short range predicitions of electrical signal data. This lead to the conclusion that Casual smoothing extrapolation is a method useful when the underlying process does not have forecasting method or is too complex to reasonably forecast but does not out perform a specially tailored forecast solution. These experiements that have been conducted cannot be considered exhaustive, and it would be advisable to do more extensive tests on each data set, adjust the parameters for both SALSA and Causal smoothing extrapolation depending on the data set and adjusting it depending on fluctuation in that set. This would achieve more accurate results which are unattainable in the confines of this paper. The most conclusive statement that can be made about these forecasting methods is that Causal smoothing extrapolation provides a more conservative estimation of future series data while losing more information about said data set. While SALSA extrapolations are in general less likely to be the actual values that occur in the future, but are much faster and have a wider range of values. With the exeption of the process which it has been designed to forecast, in which it is the superior forecasting method in all aspects. Each of these characteristics could be more useful depending on circumstances and the need for forecasting. Finally both are far better than following an average trend line when the underlying process is either too complex or unknown.