Identification of short-term and long-term time scales in stock markets and effect of structural break

TL;DR

This study analyzes short-term and long-term time scales in stock markets, considering structural breaks, using empirical mode decomposition and Hurst exponent analysis to inform investment strategies.

Contribution

It introduces a method to identify short-term and long-term time scales in stock data with structural breaks using empirical mode decomposition and Hurst exponent analysis.

Findings

Short-term time scales are within a few days to 3 months.

Long-term time scales are greater than 5 months.

Market is random in short-term and strongly correlated in long-term.

Abstract

The paper presents the comparative study of the nature of stock markets in short-term and long-term time scales with and without structural break in the stock data. Structural break point has been identified by applying Zivot and Andrews structural trend break model to break the original time series (TSO) into time series before structural break (TSB) and time series after structural break (TSA). The empirical mode decomposition based Hurst exponent and variance techniques have been applied to the TSO, TSB and TSA to identify the time scales in short-term and long-term from the decomposed intrinsic mode functions. We found that for TSO, TSB and TSA the short-term time scales and long-term time scales are within the range of few days to 3 months and greater than 5 months respectively, which indicates that the short-term and long-term time scales are present in the stock market. The Hurst exponent is $\sim 0.5$ and $\geq 0.75$ for TSO, TSB and TSA in short-term and long-term respectively, which indicates that the market is random in short-term and strongly correlated in long-term. The identification of time scales at short-term and long-term investment horizon will be useful for investors to design investment and trading strategies.

Equations

$$\Delta X_t=c+\alpha X_{t-1}+\beta t+\theta D{U}_t+\gamma D{T}_t\sum _{j=1}^k{d}_j\Delta X_{t-j}+{ϵ}_t,$$

$$D{U}_t=\{\begin{array}{ll}1,& \text{if t>TB;}\\ 0,& \text{Otherwise}\end{array}$$

$$D{T}_t=\{\begin{array}{ll}t-T_B,& \text{if t>TB;}\\ 0,& \text{Otherwise.}\end{array}$$

$${\mu }_m=\frac{1}{n}\sum _{q=1}^n{N}_{q,m}$$

$$S_m=\sqrt{\frac{1}{n}\sum _{q=1}^n(N_{q,m}-{\mu }_m{)}^2}$$

$$R/S_n=\frac{1}{P}\sum _{m=1}^P{R}_{V_m}/S_{V_m}$$

$$N{V}_i=\frac{\sqrt{\sum _{t}IM{F}_i^2(t)}}{\sum _{i=1}^N\sqrt{\sum _{t}IM{F}_i^2(t)}},$$

Full text

Stock markets, Time series, Structural break, EMD, Hurst exponent

Investors adopt different strategies in short-term and long-term depending on the investment time horizon as stock markets show different dynamics in different time scales. Hence identification of time scales in short-term and long-term is very important. The time scales have been identified using the empirical mode decomposition based Hurst exponent and normalised variance technique. The robustness of the analysis is further confirmed with Zivot and Andrews structural trend break model. Stock markets show random nature in short-term with time scales ranging from few days to nearly 3 months, and in long-term it shows long-range correlation with time scales greater than 5 months. We hope that these results may help the investors to take better decision on devising short-term and long-term trading strategies.

I Introduction

Stock market is a nonlinear, nonstationary and complex system, whose dynamical behaviour is mainly governed by the participation of different types of investor as well as various external parameters, and the dynamical behaviour is reflected in the stock price movement Bachelier (1900); Mantegna and Stanley (1999); Bouchaud and Potters (2003); Sornette (2017); Huang et al. (2003). For a long time it was believed that the price variation of stock market is generated by the random process, and subsequently random walk model was suggested to explain the movement of future price change that does not depend on the past price and is not related with economic and fundamental information Alexander (1964). Based on the Random walk theory, Efficient market hypothesis (EMH) was proposed by Fama (1965), which says that current stock price reflect all the available public information, that equally known to all investors, and hence no one can continuously make profit due to efficient nature of market Fama (1995).

Contrary to this thought, Mandelbrot showed during the period of development phase of Random theory that stock market possesses long memory and that can be described by the fractional Brownian motion Mandelbrot (1963), and subsequently several studies also suggest that market shows long-range memory Mantegna and Stanley (1999); Di Matteo (2007); Lin and Fei (2013); Stanley, Plerou, and Gabaix (2008); Kristoufek and Vosvrda (2013); Jung et al. (2008); Couillard and Davison (2005); Caporale et al. (2016); Filho and Rocha (2019), and can’t be described fully by the EMH theory LeRoy (1976); Malkiel (2003).

Survey results of the fund managers and exchange dealers of different countries and various other works show that the investors segregate market in two main investment horizons in terms of time scales, namely short-term investment horizon (STIH) where technical analysis is applied and long-term investment horizon (LTIH) where fundamental analysis is used Menkhoff (2010); Lui and Mole (1998). Here time scales for STIH and LTIH are chosen empirically. Further it has been observed that the stock price movement is different in different time scales due to the different reaction and trading strategy of investors as well as various external parameters Conrad and Kaul (1998); Jegadeesh and Titman (1993); Patzelt and Bouchaud (2018); Mahata and Nurujjaman (2019). Hence, identification of time scales for STIH and LTIH is very important for implementation of investment strategy.

Recently, in Ref. Mahata and Nurujjaman (2019) it is shown that the time scales for STIH and LTIH can be separated using empirical mode decomposition based Hurst exponent (EMDH) analysis, where, entire time series has been used for the analysis. However, it has been observed that structural change may happen in the stock data due to various factor Narayan and Smyth* (2005); Chaudhuri and Wu (2003); Worthington and Pahlavani (2007), and hence, analysing the stock data considering the structural break is very important.

In this paper we present the comparative study of the nature of stock markets in short-term and long-term investment horizon. Zivot and Andrews structural trend break model, empirical mode decomposition based Hurst exponent and variance techniques were applied to identify the short-term and long-term time scales, and we found the time scales for short-term and long-term are within the range of few days to 3 months and greater than 5 months respectively. The identified time scales show that the short-term and long-term time scales are present in the stock markets. Using Hurst exponent analysis we found that markets is random in short-term and strongly correlated in long-term.

The rest of the paper is organised as follows: Sec II discusses the method of analysis, Sec III presents stock data analysed. Sec.IV talk about results and discussion and finally Sec.V provides the conclusions.

II Method of analysis

We have used Zivot and Andrews (ZA) (1992) structural trend break model-C to identify the structural break in the stock index time series Zivot and Andrews (1992) as given below:

[TABLE]

where $\Delta$ is the first difference of the variable, $DU_{t}$ is an indicator of dummy variable for a mean shift occurring at time trend break ($T_{B}$), where $1<T_{B}<T$. $DT_{t}$ is the corresponding trend shift variable, $\epsilon_{t}$ is white noise disturbance term and k is extra regressor added to eliminate possible nuisance-parameter dependencies in the limit distributions of the test statistics caused by temporal dependence in the disturbances. $DU_{t}$ and $DT_{t}$ are defined as follows

[TABLE]

and

[TABLE]

.

The null hypothesis of Eqn. 1 is $\alpha=0$. This indicates that the time series have unit root with a drift that excludes any structural break point. Whereas, the alternative hypothesis is $\alpha<0$ represents the time series is a trend stationary with one structural break point. By using ZA test we divided the original time series (TSO) of stock data into time series before structural break (TSB) and time series after structural break (TSA). Empirical mode decomposition (EMD) technique has been applied on the TSO, TSB and TSA to identify time scales in short-term and long-term.

EMD technique is applied to decompose a signal where both nonlinearity and nonstationarity of the signal remain preserved Huang et al. (1998a, b). The main advantage of the EMD technique is that it decomposes a signal into a numbers of intrinsic mode functions (IMF) of different time scales. An IMF satisfies the following two necessary conditions: (a) the numbers of extrema and zero crossings in the signal will be equal or differ by one; (b) the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero. Detailed algorithm of calculating IMF are given below:

Lower envelope $U(t)$ and upper envelope $V(t)$ are drawn by connecting minima and maxima of the data respectively using spline fitting. 2. 2.

Mean value of the envelope $m=[U(t)+V(t)]/2$ is subtracted from the original time series to get new data set $h=X(t)-m.$ 3. 3.

Repeat the process (a) and (b) by considering $h$ as a new data set until the IMF conditions [(a) and (b)] are satisfied. Once the conditions are satisfied, the process terminates and $h$ is stored as IMF1. The IMF2 is calculated repeating the above steps (1)-(3) from the data set $d(t)=X(t)-IMF1$. When the final residual is monotonic in nature, the steps (a)-(c) are terminated and the orignal time series can be written as a set of IMFs plus trend, $X(t)=\displaystyle\sum_{i=1}^{n}IMF_{i}+residual,$ where $IMF_{i}$ represents the $i^{th}$ IMF.

The instantaneous frequency of each IMF can be estimated using the Hilbert transform define as $\displaystyle Y(t)=\frac{P}{\pi}\int_{-\infty}^{\infty}\frac{IMF(t)}{t-t^{\prime}}dt$, where $P$ is the Cauchy principle value. The frequency can be calculated by $\omega=\frac{d\theta(t)}{dt}$, where, $\displaystyle\theta(t)=tan^{-1}\frac{Y(t)}{IMF(t)}$ Huang et al. (1998a). As each IMF represents the mono-frequency component, the time scale of a particular IMF can be obtained from $\tau=1/\omega$.

Each IMF represents a signal with particular time scale. The IMF1 contains the lowest time scale present in the time series, and the IMF2 contains the second lowest time scale and so on. It can be concluded that the IMF1 fluctuates faster than the second and so on. Hence, EMD technique can be used to separate various important time scales present in a signal in the form of IMFs. As the main purpose of this work is to decompose the stock market indices with different time scales, IMFs will be very useful for this analysis. Identification of important IMF is very essential to separate the market dynamics in terms of short-term and long-term trading activities, and the identification of important IMF(s) can be done by using Hurst exponent ($H$) and Normalised variance ($NV$) analyses that are given below.

Rescaled-range (R/S) analysis technique has been applied to find $H$ exponent of a time series Hurst (1951); Mandelbrot and Wallis (1969); Mandelbrot (1972); Mandelbrot and Taqqu (1979). Let us take a time series $X=\{X_{i},i=1,2,3,\cdot\cdot\cdot,N\}$ of length N and divided the series into P subperiod of length n in such a way that $P\times~{}n=N$. Let us define each subperiod as $V_{m}$, where $m=1,2,3,\cdot\cdot\cdot,P$ and each element is $N_{q,m}$, where $q=1,2,3,\cdot\cdot\cdot,n$. Average and the standard deviation of each subperiod are expressed as

[TABLE]

and

[TABLE]

respectively. The cumulative departure of the time series is given by $Y_{q,m}=\displaystyle\sum_{j=1}^{q}(N_{j,m}-\mu_{m})$, where $q=1,2,3,\cdot\cdot\cdot,n$. Range of the cumulative departure from the mean can be estimated by using $R_{V_{m}}=max(Y_{q,m})-min(Y_{q,m})$, where $1\leqslant q\leqslant n$. Individual subperiod range can be rescaled or normalized by dividing the standard deviation. So, rescaled range (R/S) can be represented as

[TABLE]

. In general, $(R/S_{n})=c\times n^{H}$, where c is the constant and $H$ is the Hurst exponent. $H$ can be estimated from the slope of $log(R/S_{n})$ versus $log(n)$. For a random time series, $H$ is 0.5, and for persistent and anti-persistent data, H is greater than 0.5, and less than 0.5 respectively.

$NV$ is an important statistical tool to analyse financial data that can be applied to separate signals with respect to their energy. It has been observed that the energy of a time series is mostly concentrated in its important IMF(s) Chatlani and Soraghan (2012); Zao, Coelho, and Flandrin (2014). The concentration of energy can be quantified by estimating the normalised variance $NV$ of the IMFs, and hence $NV$ can be used to identify the important IMFs. The $NV$ of an IMF can be calculated using the formula

[TABLE]

where, $N$ is the total number of IMF.

III stock data analysed

In the present study, we have analysed the different stock indices and stock price of different countries such as (1) S&P 500 (USA), (2) Nikkei 225 (Japan), (3) CAC 40 (France), (4) IBEX 35 (Spain) (5) HSI (Hong Kong), (6) SSE (China), (7) BSE SENSEX (India), (8) IBOVESPA (Brazil), (9) BEL 20 (Euro-Next Brussels), (10) IPC (Mexico), (11) Russel2000 (London),(12) TA125 (Israel) and companies like (13) IBM (USA), (14) Microsoft (USA), (15) Tata Motors (India), (16) Reliance Communication (RCOM) (India), (17) Apple (USA), (18) Reliance Industries Limited (RIL) (India). All the above data are taken from December 1995 to July 2018, and downloaded from yahoo finance yah . We have chosen the above indices and company mainly because of they are the leading global index. Figs. 1 show the stock indices of (a) S&P 500 index (USA), (c) SSE Index (china), (e) Nikkei 225 index (Japan), and companies (b) Microsoft (USA), (d) IBM (USA) and (f) Tata Motors (India). The analysis of the data is given in the next section.

IV Results and Discussion

In order to understand the behaviour of stock markets and stock price of various indices and companies at different time scales, we have applied the empirical mode decomposition (EMD) technique to decompose the original time series (TSO), time series before structural break (TSB) and time series after structural break (TSA) of stock index and price into IMFs of different time scales that is presented in the Subsec. IV.1.

IV.1 EMD analysis

Fig. 2 (a), (b) and (c) show nine intrinsic mode functions (IMFs) along with the residue separately of TSO, TSB and TSA of the S&P 500 index respectively. Each IMF portrays the simple mono-frequency component of the stock data. All the IMF1 in Fig. 2 (a), (b) and (c) represents the mode with highest frequency, and the frequency gradually decreases with the increase of IMF numbers for TSO, TSB and TSA. Figs. 2 (a)-(c) show that the IMFs obtained from TSO [2 (a)], TSB [2 (b)] and TSA [2 (c)] are similar in nature. The last plots in Fig. 2 (a), (b) and (c) are the residue of the original time series, time series before and after structural break. The residue are the representation of the average trend of original time series, time series before and after structural break of the S&P 500 index. In order to understand the nature of all similar IMFs of original time series, time series before and after structural break, detailed analysis using Hurst technique have been carried out in Subsec. IV.2.

IV.2 Hurst exponent analysis

Hurst exponent ($H$) of all the decomposed IMFs has been calculated using rescaled-ranged analysis presented in Sec. II. Fig 3 shows the typical plots of $\ln(R/S)$ vs. $\ln(lag)$ of IMF1 to IMF9 and residue. Slope of the each plot is basically $H$ value of the corresponding IMF. From the Fig 3 it is clear that slope of IMF1 to IMF5 is nearly 0.5 and suddenly increases. The value of $H\sim 1.0$ for residue of the index.

Figs. 4 (a)-(f) show the value of $H$ of the IMF1 to IMF9 of the original time series, the time series before and after structural break of S&P 500, Nikkei 225, CAC 40, IBX35, HSI, and SSE respectively. The plots show that the value of $H$ is around 0.5 for IMF1 to IMF5 of the original time series, the time series before and after structural break. The value of $H$ increases suddenly to $\sim 0.75$ for IMF6 and gradually increases till the last IMF for the all three time series. The $H$ value for IMF6 to IMF9 is between 0.75 to 1.0. It is already shown that $H\sim 1$ for residue of all the data. Similarly, Figs. 5(a)-(f) show the $H$ of the IMF’s of BSE SENSEX, IBOVESPA, BELL 20, IPC, Russell 2000 and TA125 respectively, and Figs. 6(a)-(f) show the $H$ of the IMF’s of IBM, Microsoft, Tatamotors, RCOM, Apple and RIL respectively. It is pertinent to mention that we were not able to calculate $H$ exponent of CAC 40 for TSB, Microsoft for TSA and RCOM for TSB, RIL for TSA because of short data lengths.

From Figs. 4,  5 and  6 it is clear that the value of $H$ is around 0.5 for IMF1 to IMF5 of all the indices and companies stock prices. We can conclude from the $H$ exponent values that the IMF1 to IMF5 are random in nature. Therefore, stock price represented by IMF1 to IMF5 is random. The $H$ value between 0.75 to 1 for IMF6 to IMF9 and residue indicates that they are having long-range correlation which in turn means less volatility and less roughness. Therefore, stock price represented by IMF6 to IMF9 and residue is persistent in nature.

In order to separate the dynamics, we have constructed two time series, $X_{ST}(t)=\displaystyle\sum_{i=1}^{5}~{}IMF_{i}$ adding IMF1 to IMF5 and $\displaystyle X_{LT}(t)=[\displaystyle\sum_{i=6}^{9}~{}IMF_{i}+Residue]$ for the original time series, the time series before and after structural break of all the stock index and data respectively. Figs. 7 (a)-(c) shows the reconstructed short-term $X_{ST}(t)$ and long-term $X_{LT}(t)$ time series. Finally we have calculated the $H$ exponent of the reconstructed time series $X_{ST}(t)$ and $X_{LT}(t)$ of three time series which is plotted in Fig 7. The plot  8 shows that $H\sim 0.5$ for all reconstructed time series $X_{ST}(t)$, and $H\sim 0.97$ for $X_{LT}(t)$. The results show that two reconstructed time series $X_{ST}(t)$ and $X_{LT}(t)$ actually separates the random and long-range dynamics in the system. It is pertinent to mention that $X_{LT}(t)$ represents the collective fundamentals of the companies as stated in Ref. Mahata and Nurujjaman (2019)

In order to identify the time scales in two types of dynamics, we have estimated the time scales of all the IMFs of the original time series, the time series before and after structural break of all the stock index and data. Fig. 9 (a), (b) and (c) show the time scales versus $H$ exponent for TSO, TSB and TSA respectively calculated from their IMFs. Fig. 9 (a), (b) and (c) show that the $H\sim 0.5$ for the time series with time scales few days to 3 months, i.e., for IMF1 to IMF5. Therefore, it may be concluded that the markets in ITH of few days to 3 months are random in nature. The value of $H$ suddenly increases to 0.75 for the time scales 5-7 months for TSO, 3.0-7 months for TSB and 3.7-7 months for TSA, from IMF6 as shown in the same figures, and with increase in time scales $H$ approaches 1. As $H>0.5$ indicates the long-range correlation, we can conclude that the stock markets in time scales greater than five months show long-range correlation.

IV.3 Normalised variance analysis

It has been observed that the energy of a time series is mostly concentrated in its important decomposed IMFs Chatlani and Soraghan (2012); Zao, Coelho, and Flandrin (2014), and the energy can be quantified by calculating the normalised variance ($NV$) of the IMFs that has already been described in Sec II.

Figs. 10(a)-(c), (d)-(e) and (f)-(h) show the normalised variance ($NV$) of the IMF’s of TSO, TSB and TSA of the stock indices and company price respectively. Fig. 10(a) shows IMF7 with time scale 2.5 months is important; Fig. 10(b) shows that IMF8 is important for several indices and companies as given in plot legends of the Figure, where time scale of the IMF is around 0.8 to 2 yrs. Fig. 10(c) shows that IMF9 is important for other indices and companies shown in the legends. Similar results have been obtained for TSB and TSA time series as shown in the plots (d)-(h). The results show that in long-term investment horizon, different time scales may be important depending on the market dynamics of particular index.

V Conclusions

Our study reveals that the stock markets possesses the long-range correlation and randomness depending on time scales. We have separated market dynamics into short-term and long-term by using empirical mode decomposition based Hurst exponent analysis. Decomposition has been done of each index and company into nine intrinsic mode functions (IMF) and a residue. Each of the IMFs have different time scales and frequency. We found that for IMF1 to IMF5 $H\sim 0.5$, and for IMF6 to IMF9 and residue $H\geq 0.75$. We have also obtained similar results for the TSO, TSB and TSA of all the stock index and data obtained by applying Zivot and Andrews structural trend break model. The results show that market is random with time scales from few days to 3 months and shows long-range correlation with time scales $\geq 5$ months.

Our finding gives an idea about the behavior of the stock markets which may be applicable for both the short-term and long-term investors. Our results suggest that short-term trading is very risky due to presence of randomness in shorter investment horizon. However, in long-term trading, risk is less because of the presence of long-range correlation in the stock markets.

Acknowledgment

We would like to acknowledge Jean-Philippe Bouchaud for some valuable discussion and suggestions.