A New Stock Market Valuation Measure with Applications to Retirement Planning

TL;DR

This paper introduces a new stock market valuation measure derived from a generalized cyclically adjusted price-earnings ratio, challenging efficient market theory and providing insights for retirement planning.

Contribution

It generalizes the CAPE ratio by modeling earnings and wealth dynamics, offering a novel valuation metric and implications for long-term return predictions.

Findings

Disproves the Efficient Market Hypothesis.

Long-run total returns equal earnings growth plus 4.6%.

Provides a framework for retirement withdrawal strategies.

Abstract

We generalize the classic Shiller cyclically adjusted price-earnings ratio (CAPE) used for prediction of future total returns of the stock market. We treat earnings growth as exogenous. The difference between log wealth and log earnings is modeled as an autoregression of order 1 with linear trend 4.6% and Gaussian innovations. Detrending gives us a new valuation measure. Our results disprove the Efficient Market Hypothesis. Therefore, long-run total returns equal long-run earnings growth plus 4.6%. We apply results to retirement planning. A withdrawal process governs how a retired capital owner withdraws a certain fraction of wealth annually. We study the long-term behavior of such processes.

Tables

Table 1. Table 1. Regression Results ( 4 ).

coefficient	point estimate	$p$-value	[2.5%, 97.5%] CI
$\alpha$	0.1008	0.2%	[0.038, 0.164]
$\beta$	0.0070	0.1%	[0.003, 0.011]
$\gamma$	0.1545	0.1%	[0.066, 0.243]

Table 2. Table 2. Regression Results: Fundamentals as 1-year earnings (1E) or dividends (1D), 5-year averaged trailing earnings (5E) or dividends (5D)

Statistics	1E	1D	5E	5D
$g$	1.9%	1.6%	1.9%	1.5%
$\sigma \left(G\right)$	28%	11%	8.1%	5%
$\sigma \left(\epsilon \right)$	26%	19%	18%	17%
$R^2$	16%	12%	7.8%	9.1%
$h$	0.32	0.11	0.36	0.25
$b$	0.85	0.75	0.86	0.82
$c$	4.6%	5%	4.5%	5%
$\gamma$	0.33	0.25	0.15	0.18
$T\left(\gamma \right)$	5.4	3.4	4.5	3.8
$H\left(2023\right)$	-0.1	-0.24	0.0025	-0.09
$\rho \left(P\right)$	-73%	-43%	-29%	-26%
$\rho \left(S\right)$	-60%	-41%	-20%	-19%
$p\left(SW\right)$	0.2%	32%	45%	3%
$p\left(JB\right)$	$2\cdot {10}^{-7}$	29%	33%	0.49%
$p\left(O5\right)$	81%	4.7%	10%	3.5%
$p\left(O10\right)$	37%	0.4%	14%	5.1%
$p\left(A5\right)$	31%	7%	11%	11%
$p\left(A10\right)$	31%	15%	25%	16%

Equations

$$U(t)=\ln V(t)-\ln F(t).$$

$$H(t)=U(t)-ct.$$

$$H(t)-h=b(H(t-1)-h)+\epsilon (t),\epsilon (t)\sim \mathcal{N}(0,{\sigma }^2)\text{mbox}i.i.d.$$

$$\ln V(t)-\ln F(t)-h-ct=b(\ln V(t-1)-\ln F(t-1)-h-c(t-1))+\epsilon (t).$$

$$Q(t)-G(t)=\alpha +\beta t-\gamma (\ln V(t-1)-\ln F(t-1))+\epsilon (t),$$

$$b=1-\gamma ,c=\frac{\beta }{\gamma },h=\frac{\alpha -c}{\gamma }.$$

$$b=0.86,c=4.5\%,h=\mathrm{0.36.}$$

$$G(t)=\ln \frac{F(t)}{F(t-1)},t=1,2,\dots$$

$$Q(t)=H(t)-H(t-1)+c+G(t),t=1,2,\dots$$

$$\overline{Q}(T)=\frac{1}{T}\left[Q(1)+\dots +Q(T)\right].$$

$$V(T)=\exp (Q(1)+\dots +Q(T))=\exp (T\overline{Q}(T)).$$

$$\overline{G}(T)\text{mathchar}58=\frac{G(1)+\dots +G(T)}{T}=\frac{1}{T}\ln \frac{F(T)}{F(0)}\to g,T\to \infty .$$

$$\mathbb{P}(G(t)\in A\mid G(t-1)=x,\epsilon (t)=y)={\int }_A\mathbf{p}(x,y,z)\mathrm{d}z.$$

$$\overline{Q}(T)=c+\frac{H(T)-H(0)}{T}+\overline{G}(T),$$

$$V(T)=\exp \left[cT+H(T)-H(0)\right]\frac{F(T)}{F(0)}.$$

$$\frac{1}{T}(\ln V(t)-\ln F(t))\to c\text{mbox}almostsurely.$$

$$\overline{Q}(T)\to c+g\text{mbox}almostsurelyasT\to \infty .$$

$$d_{\mathrm{TV}}(P,Q)=\underset{A\subseteq {\mathbb{R}}^2}{\sup }|P(A)-Q(A)|,$$

$${\mathcal{W}}_p(P,Q)=\underset{X\sim P,Y\sim Q}{\inf }\mathbb{E}\left[|X-Y{|}^p\right].$$

$$\mathbb{P}$$

$$=\mathbb{P}\left((G(t),H(t))\in A\mid H(0),G(1),H(1),\dots ,G(t-1),H(t-1)\right).$$

$$\sqrt{T}(\overline{Q}(T)-c-g)\to \mathcal{N}(0,{\rho }^2),T\to \infty .$$

$$V_W(t)=V_W(t-1)e^{Q(t)}(1-W(t)),t=1,2,\dots ;V_W(0)=1.$$

$$\frac{1}{T}\sum _{t=1}^{T}W(t)\to w\text{mbox}a.s.asT\to \infty .$$

$$W(t)=1-\exp (u+G(t)).$$

$$R(t)=△M(t)+\Gamma (t)+\Delta (t),$$

$$\overline{Q}(T)$$

$$=c+\frac{H(T)}{T}-\frac{H(0)}{T}+\frac{1}{T}\left(G(1)+G(2)+\dots +G(T)\right).$$

$$\frac{1}{T}\left(\ln V(t)-\ln F(t)\right)=\frac{1}{T}\left(cT+H(T)-H(0)\right)=c+\frac{H(T)-H(0)}{T}.$$

$$\frac{H(T)-H(0)}{T}\to 0,T\to \infty .$$

$$\frac{H(1)+\dots +H(T)}{T}\to c,T\to \infty .$$

Full text

Acknowledgements. The author is thankful to UNR students Akram Reshad, Taran Grove, Erick Luerken, Tran Nhat, and Michael Reyes for useful discussion. The author thanks the Department of Mathematics and Statistics at the University of Nevada, Reno, for welcoming atmosphere for research. The author thanks the anonymous referee for useful comments. The author received no funding and declares no conflict of interest.

1. Introduction

1.1. Index, earnings, and dividends

Forecasting future stock market returns is a major area of study in financial mathematics. We measure stock market by the Standard & Poor (S&P) 500 and its predecessors. Robert Shiller collected USA annual stock market data from 1871, which we use in this article. The index (real, that is, inflation-adjusted) is in Figure 1 (A). The wealth (real) is in Figure 1 (B). It grows much faster than the index, because it includes reinvested dividends. Dividends are part of earnings (net income) paid by companies to shareholders (usually quarterly or annually). The rest of earnings are retained by companies for other purposes. The combined dividends/earnings of all S&P 500 companies (including the ones with negative earnings or zero dividends) form S&P 500 annual dividends/earnings. Thus shareholders get returns in two ways: (a) stock/index price increases; (b) dividends payout.

Annual earnings and dividends are in Figure 2 (A). Averaged 5-year earnings and dividends are in Figure 2 (B). One can see that in both graphs, in almost all years, earnings exceed dividends. That is, the upper graph is earnings, and the lower graph is dividends. However, earnings of some companies can be negative. This creates a problem called aggregation bias: Adding negative earnings of these companies to positive earnings of other more successful companies understates the total. Owning companies with negative earnings does not diminish your claims on positive earnings of other companies.

1.2. P/E ratios

There is a long tradition of using different versions of a price-to-earnings (P/E) ratio. A stock market or portfolio with high P/E ratio is deemed overpriced, with bearish growth prospects. Conversely, a stock market of portfolio with low P/E ratio is considered attractively underpriced. Robert Shiller designed a cyclically adjusted price-earnings (CAPE) ratio which takes 10-year averaged inflation-adjusted earnings in the denominator, [6, 18]. See a discussion in [19, Chapter 11]. A related concept is value investing which applies the same concept to individual stocks, see the classic article [8] and [19, Chapter 12]. Instead of surveying all existing literature, we refer the reader to the books [10, 18, 19] cited above, as well as [1, 3, 13, 14, 16].

We consider both annual S&P 500 earnings and dividends data, as well as 5-year trailing averages of each. We average to smooth out business cycle fluctuations. During recessions, earnings (and, to a lesser extent, dividends) temporarily plummet, but the impact upon long-term earnings prospects is small. Thus P/E ratio can increase during recessions, but this does not mean that the stock market is overpriced. The CAPE ratio does not have this problem, because it uses averaged earnings. We mentioned aggregation bias above. It is most pronounced during recent recessions: the dot-com crash and the financial crisis. (This remark does not apply to dividends, because they are always nonnegative.) However, we have only this aggregated data. This is an additional reason to average trailing data.

We can also use dividends instead of earnings to compute these ratios. As mentioned earlier, some earnings are paid as dividends, others are retained by companies for other uses. Thus dividends are usually less than earnings. On the other hand, earnings can be negative, but dividends are always nonnegative.

This CAPE ratio has significant predictive power for future total real stock market returns (including dividends and price changes). We use the 5-year instead of the 10-year version here, to have slightly more data points (148 instead of 143). However, these trailing versions are not very different. The price-earnings and price-dividend ratios are shown in Figure 3: the classic versions (without averaging) in Figure 3 (A) and the versions with 5-year averaging in Figure 3 (B). Since earnings exceed dividends, as mentioned before, in both graphs, the price-dividend ratios exceed the price-earnings ratios for the same year.

1.3. The new valuation measure

There is evidence that companies prefer to distribute earnings to shareholders via buybacks, rather than dividends. This pushes price upward and increases these P/E ratios, but does not necessarily affect total returns (which combine both price changes and dividend payouts). We consider another valuation measure.

Let $V(t)$ be the real wealth in (January of) year $t+1$ given $V(0)=1$. Then $Q(t)=\ln(V(t)/V(t-1))$ are total real returns (including dividends) in year $t$. Averaged annual returns 1871–2022 are $6.7\%$. Let $F(t)$ be the fundamentals (earnings or dividends, annual or trailing 5-year) in year $t$. Then $G(t)=\ln(F(t)/F(t-1))$ is the (logarithmic) growth in fundamentals. Averaged annual growth is approximately $2\%$. Next, consider

[TABLE]

It grows annually at rate $c\approx 6.7\%-2\%$, and can be modeled as an autoregression (AR) of order $1$ with linear trend $ct$. Namely, first detrend $U$ by subtracting

[TABLE]

Then we model $H$ from (2) as an AR(1) with Gaussian innovations $\varepsilon$:

[TABLE]

Here, $\beta\in(0,1)$; a classic Student $t$-test (which we can apply because the innovations are Gaussian) shows that $\beta\neq 1$ significantly. Therefore, $H$ from (2) is not a random walk. This disproves the Efficient Market Hypothesis, because we have predictability in returns via (1).

We use $H$ as the new valuation measure, shown in Figure 4. These two valuation measures (CAPE and $H$) are close until the last 20 years, then they diverge. You can see two measures on Figure 7. As of January 2023, the CAPE shows that the market is overpriced, but $H$ shows that it is underpriced. As mentioned earlier, this is due to recently changed dividend payout policy. They give opposite results: The new measure indicates that currently (as of January 2023) the U.S. stock market is undervalued and future returns are higher than historical averages. The old measure indicates the opposite.

1.4. Disclaimers

This regression (3) has $R^{2}=8\%$. A statistican could raise questions about relevance and importance of this regression. However, it is normal to get such low $R^{2}$ for financial prediction. The stock index movement is very close to a random walk. Still, here we got statistically significant results, according to Student $t$-tests.

We stress that the stock market is underpriced only in the long run. There is a lot of noise and only a little signal. The valuation measure shows long-term potential. But the next year or two might as well be different. Like in 1999, the stock market was widely overpriced according to any measure, Shiller CAPE or our valuation measure. however, in 2000 it still zoomed upwards.

1.5. Retirement processes

Consider a retiree, who owns a capital and invests it in S&P 500 index. The retiree withdraws annually a certain fraction of wealth. These withdrawals depelete to some extent the capital, so the resulting wealth process grows slower than the original process $V$. A classic example of a withdrawal strategy is the 4% rule, which exists in two versions: proportional (4% of current wealth) and constant (4% of inflation-adjusted initial wealth). In this article, we study the proportional versions. A withdrawal strategy is called sustainable if the wealth process almost surely converges to infinity. In Section 4, we state and prove sufficient conditions for the withdrawal strategy to be sustainable or not sustainable. In particular, the 4% proportional rule is, in fact, sustainable.

1.6. Organization of the article

We perform this statistical analysis for (3) assuming unknown $c$ in Section 2. We consider four versions of fundamentals $F$: earnings or dividends, each either annual or 5-year trailing averaged versions. Next, in Section 3, we state and prove long-term stability results, assuming long-term stability of earnings or dividends growth, and using the long-term stability for the AR(1) in (3): Strong Law of Large Numbers in Theorem 15 and the Central Limit Theorem in Theorem 4. In Section 4, we study retirement planning. We consider withdrawal strategies: A retiree needs to withdraw certain amounts each year from wealth. We state Theorem 5 for proportional withdrawal strategies (fractions of current wealth). Section 5 is devoted to discussions: detailed explanations of economic and financial sense behind this statistical analysis. Section 6 is devoted to conclusions and suggestions for future research. The Appendix contains proofs of all results. The GitHub repository asarantsev/IDY contains our Python code and data.

2. Statistical Analysis

2.1. Basic analysis

Combine (1), (2), (3) into one linear regression:

[TABLE]

We can rewrite (4) as follows:

[TABLE]

where the old and new coefficients are related as follows:

[TABLE]

First, fit the regression (5) for 5-year averaged trailing real earnings. See Table 1 for results. As for the regression residuals $\varepsilon(t)$, in Figure 5 we see their quantile-quantile (QQ) plot vs the normal distribution, and the autocorrelation function (ACF) for $\varepsilon(t)$ and $|\varepsilon(t)|$. Based on this analysis (and the statistical tests found later in Table 2), it is reasonable to conclude that $\varepsilon(t)\sim\mathcal{N}(0,\sigma(\varepsilon)^{2})$ are i.i.d. Gaussian.

In Table 1 all coefficients are significantly different from zero. Of particular importance is the coefficient $\gamma$, which is responsible for mean reversion. The parameter $b$ is between [math] and $1$. This implies that the autoregression (4) is mean-reverting, that is, has long-term stability. Plugging point estimates for $\alpha,\beta,\gamma$ from Table 1 into (6), we get:

[TABLE]

These point estimates in (7) are important. Below, we discuss each of these parameters.

2.2. Economic meaning of the autoregression coefficients

The parameter $b$ is responsible for mean reversion: Since $b\in(0,1)$, this autoregression is indeed mean-reverting, not a random walk. As discussed, we reject the unit root hypothesis, which in this context means $b=\pm 1$. The closer $b$ to zero, the stronger the mean reversion is.

The parameter $c$ is the long-run average of the implied dividend yield $\Delta$. This can be viewed as the long-term value of the second component of returns. As discussed earlier, this number matches the return on the capital in traditional societies with zero economic and earnings growth. Together with earnings growth (which has long-term average $g=1.9\%$), this gives us total real returns. The latter have long-term average $c+g=4.5\%+1.9\%=6.4\%$; see Theorem 15.

Finally, $h$ is the long-term average of the new valuation measure. Comparing this measure with $h$, one can find whether the stock market is over- or underpriced. The current (as of January 2023) value of $H$ is equal to $H(147)=0.25<h$. Thus the current stock market is underpriced, and future total real returns, given the same real earnings growth as in the last 152 years, are higher (than the overall long-term average $c+g$). As mentioned in the Introduction, this promise of higher returns is in the long-run. There is a lot of noise in the markets, and next year returns can easily be very negative.

2.3. Four versions of fundamentals

In the following summary table, we present the results of regression analysis (5) for four versions of the fundamentals: earnings/dividends, each either annual or 5-year trailing averaged.

In the first part, we provide basic analysis. First, $m(G)$ is the empirical mean of $G$, and $\sigma$ with indices $G$ and $\varepsilon$ are standard deviations of $G$ and $\varepsilon$. Then we provide the $R^{2}$ for this regression, and the values of $b,c,h$. Next, we provide the value of the coefficient $\gamma$ from (5) and its Student $t$-statistics $T(\gamma)$. The current value of the valuation measure (as of January 2023) is denoted by $H(2023)$. Comparing it with the long-term value $h$, we find whether the market is over- or undervalued. In the second part, we perform analysis of regresson residuals $\varepsilon(t)$. The $p$-values with indices SW, JB are for Shapiro-Wilk and Jarque-Bera normality tests for $\varepsilon(t)$. The $p$-values with indices O5, O10 are for Ljung-Box test for $\varepsilon(t)$ with 5 and 10 lags, respectively. The $p$-values with indices A5, A10 are for Ljung-Box test for $|\varepsilon(t)|$ with 5 and 10 lags, respectively. Finally, $\rho$ with indices $P$ and $S$ stands for Pearson (classic) and Spearman (robust) correlations between $G(t)$ and $\varepsilon(t)$.

Real earnings/dividends growth terms $G(t)$ (in each of the four versions) are not white noise (judging by the ACF) and not Gaussian (judging by the Shapiro-Wilk and Jarque-Bera normality tests). This is why we compute Spearman correlations between $G(t)$ and $\varepsilon(t)$.

In Table 2, we conclude that $R^{2}$ is the best for one-year versions. But only for 5-year averaged trailing earnings (5E) we can conclusively accept that residuals are i.i.d. Gaussian. This is important: we model $U$ as AR(1) with linear trend (which requires residuals to be white noise), and apply classic Student tests (which requires normality of residuals).

A low value of $R^{2}$ in all these cases is not a judgement of the quality of regression analysis. This is simply a reflection of the observation that stock index returns have a lot of randomness and little predictability. However, for all these regressions the coefficient $\gamma$ is significant: The Student (Gosset) $t$-statistics is always significantly greater than $3$. This points to mean reversion and returns predictability, and disproves the Efficient Market Hypothesis.

Additionally, in each case, the current valuation measure is less than the long-term average $h$. Thus the market is undervalued under each of these analyses. The long-run returns are lower than the theoretical average $c+g$.

Standard deviation of total returns is 17%. If we treat fundamentals growth as exogenous, we must choose the version with the smallest standard deviation. The less this standard deviation is, the more the variance reduction is. This version is the 5D with standard deviation 5%. The next smallest one is 5E with standard deviation 8.1%.

2.4. 5- and 10-year versions

Why do we use the 5-year averaged trailing earnings and not the 10-year version, as suggested by Shiller’s research? He justified this choice of 10 years by the observation that the business cycle (from peak to crash and back to the next peak) lasts around a decade. Thus a ten-year averaging window takes care of this. However, in our research we found that the 5-year version does a job very well: The valuation measures computed using the 5- and 10-year earnings averages are almost identical (correlation 93%), and in fact the 10-year version actually has $R^{2}=6.4\%$ in the autoregression (5), as opposed to $R^{2}=7.8\%$ for the 5-year version. Thus 5 years, it turns out, is quite enough to smooth out annual earnings fluctuations. Incidentally, the 5- and 10-year Shiller CAPE ratios are almost identical too (correlation 95%). But the 5-year version gives us more data points.

2.5. Why the new valuation measure is superior to Shiller CAPE

The classic way to use CAPE for index returns prediction is to perform a simple linear regression of next year’s total real returns upon the CAPE value for this year. However, this results in a failure on multiple levels. The $R^{2}=2\%$, much lower than $R^{2}=8\%$ for the new valuation measure. The $p$-value for the slope is $0.08$, thus we fail to reject the null hypothesis (that the CAPE and next year’s returns are uncorrelated). Finally, and most importantly, the residuals fail both normality tests and white noise tests. They exhibit strong autocorrelation.

3. Model Statement and Theoretical Results

In this section, we create a time series model based on the statistical analysis in the previous section. We operate on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_{0},\mathcal{F}_{1},\ldots),\mathbb{P})$. We state the following well-known definitions from Stochastic Processes theory.

Definition 1**.**

A real-valued discrete-time stochastic process $X=(X_{0},X_{1},\ldots)$ is called adapted with respect to the given discrete-time filtration if $X_{t}$ is $\mathcal{F}_{t}$-measurable. The process $X$ has a stationary distribution $\pi$ if $X_{t}\sim\pi$ for all $t=0,1,2,\ldots$

Gather and reverse-engineer the equations from the previous section. Starting from $\varepsilon(t)$ and $F$, we define $G$, $H$, and $Q$.

Definition 2**.**

Take a positive adapted stochastic process $F=(F(0),F(1),\ldots)$ which we call the fundamental process. Take another adapted white noise sequence $\varepsilon(t)\sim\mathcal{N}(0,\sigma^{2})$ for $t=1,2,\ldots$ Here, $\varepsilon(t)$ is independent of $\mathcal{F}_{t-1}$. Fix constants $b\in(0,1),c>0$, and long-term valuation measure $h$. Define valuation measure $H$ using the equation (4), with $H(0)$ an $\mathcal{F}_{0}$-measurable random variable. Fundamental growth $G(t)$ for year $t$ is defined by

[TABLE]

and the total real returns $Q(t)$ for year $t$ are defined as

[TABLE]

Definition 3**.**

Average annual total real returns for $T$ years are defined as

[TABLE]

Real wealth by year $T$ (starting from wealth $1$ at year [math]) is

[TABLE]

We do not require $\varepsilon$ and $G$ to be independent or uncorrelated. Together, Definitions 9 and 11 form a discrete-time model. This is the main model of this article. In the rest of this section, we analyze this model. For some of our theorems, we impose additional assumptions. The first is the Strong Law of Large Numbers (SLLN) of annual real earnings growth terms. It does not necessarily imply that the process is mean reverting, for example an autoregression: This could be a random walk.

Assumption 1**.**

The process $G$ satisfies the SLLN: almost surely,

[TABLE]

The next assumption is the Central Limit Theorem (CLT) for the real earnings growth.

Assumption 2**.**

The process $G$ satisfies $\sqrt{T}(\overline{G}(T)-g)\to\mathcal{N}(0,\rho^{2})$ in law as $T\to\infty$.

Next, we state a Markovian assumption: that real earnings growth term $G(t)$ for time $t$ depends not on the entire history $G(1),\ldots,G(t-1)$ and $\varepsilon(1),\ldots,\varepsilon(t)$, but instead only on $G(t-1)$ and $\varepsilon(t)$. We also require continuity, since this is important for later stability proofs. We think this continuity assumption is reasonable and is usually easily satisfied.

Assumption 3**.**

For a transition density $\mathbf{p}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{R}^{3}\to(0,\infty)$ continuous in $(x,y)$,

[TABLE]

Finally, we state an assumption of bounded moments, necessary for convergence proofs.

Assumption 4**.**

$\sup_{t\geq 1}\mathbb{E}[G^{p}(t)]<\infty$ and $\mathbb{E}[H(0)|^{p}<\infty$ for some $p>1$.

Below are main results. The proofs are in the Appendix. First, a technical lemma.

Lemma 1**.**

Average annual total real returns for $T$ years are given by

[TABLE]

and wealth at time $T$ is given by

[TABLE]

Our next result is the Strong Law of Large Numbers (SLLN) for total real returns. It follows from Assumption of the SLLN for real earnings growth, as well as mean reversion of the autoregressive process.

Theorem 1**.**

Under Assumption 1, we have the following asymptotic results: as $T\to\infty$,

[TABLE]

Average total real returns satisfy the convergence statement:

[TABLE]

Further, we state ergodicity (long-term stability) for $(G,H)$. Let us introduce some definitions. For probability measures $P$ and $Q$ on $\mathbb{R}^{2}$, define the total variation distance:

[TABLE]

and the Wasserstein distance of order $p\geq 1$:

[TABLE]

In (17), the inf is taken over all couplings: pairs of random variables $(X,Y)$ such that $X\sim P$ and $Y\sim Q$. Both these functions are indeed distances: (16) is on the set $\mathcal{P}$ of all probability measures on $\mathbb{R}^{2}$, and (17) is on the set of all $P\in\mathcal{P}_{2}$ with finite $p$th moment. Finally, a discrete-time Markov process $X=(X(0),X(1),\ldots)$ on the state space $\mathbb{R}^{2}$ has a stationary distribution $\Pi$ (a probability measure on $\mathbb{R}^{2}$) if $X(0)\sim\Pi$ implies $X(1)\sim\Pi$ (hence $X(t)\sim\Pi$ for all $t$).

Theorem 2**.**

Under Assumption 3, the process $(G,H)$ is Markov: For $A\subseteq\mathbb{R}^{2}$ and $t$,

[TABLE]

Theorem 3**.**

Under Assumption 3 and 4,

(a) the process $(G,H)$ has a unique stationary distribution $\Pi$ on $\mathbb{R}^{2}$ with positive Lebesgue density on $\mathbb{R}^{2}$, with finite $p$th moments, and a Gaussian marginal for $H$: $\Pi_{H}=\mathcal{N}(h,\sigma^{2}_{\Pi})$, where $\sigma^{2}_{\Pi}=\sigma^{2}/(1-b^{2})$;

(b) regardless of the initial distribution, the distribution of $(G(t),H(t))$ converges to $\Pi$ as $t\to\infty$ in the total variation and in the Wasserstein distance of order $p$.

Finally, we state the CLT for total real returns: In the long run, wealth is a lognormal random variable. This follows from the CLT assumption for the real earnings growth series.

Theorem 4**.**

Under Assumptions 1,  2, we have weak convergence

[TABLE]

4. Retirement Planning

A major problem in finance is retirement planning: How much a retiree can withdraw per year to keep income more or less stable, and still preserve wealth? Wealth funds and university endowments face a related problem; the difference is that, unlike the retiree, they wish to preserve the wealth forever, and not for 20–30 years. A classic withdrawal rule is 4%. It exists in two versions: Constant: Withdraw inflation-adjusted 4% of initial wealth; Proportional: Withdraw 4% of current wealth.

The first rule ensures constant stream of payments, but could deplete wealth up to zero (ruin, or bankruptcy). A financial planner must ensure that the ruin probability is small. The second rule ensures that there is no ruin, but income could fluctuate (when markets crash, income decreases). In this article, we consider only the proportional rule. We generalize it for the case of changing (and possibly random) proportional withdrawal rate.

Definition 4**.**

A withdrawal process $W=(W(0),W(1),\ldots)$ is a discrete-time stochastic process with values in $(0,1)$. Its corresponding wealth process $V_{W}$ is defined as

[TABLE]

For example, if total real return $Q(t)=7\%$, and withdrawal rate $W(t)=4\%$, then wealth from $t-1$ to $t$ is multiplied by $e^{0.07}$ from total real return and is multiplied by $0.96$ from withdrawal. Thus the overall multiplication factor is $e^{0.07}\cdot 0.96\approx 1.03$. If withdrawal process is not constant, we define an asymptotic rate: a long-term limit.

Definition 5**.**

A withdrawal process $W$ has asymptotic rate $w$ if it satisfies

[TABLE]

For any withdrawal process, $V_{W}(t)>0$ almost surely for all $t$. But still, $V_{W}(t)$ could decrease to small values, which is, of course, undesirable. We define a condition under which this does not happen.

Definition 6**.**

A withdrawal process is called sustainable if $V_{W}(t)\to\infty$ a.s. as $t\to\infty$.

Our main result is as follows: We prove sufficient conditions for sustainability, and other sufficient conditions for non-sustainability. There is a gap between these two results, because $c+g>1-e^{-c-g}$, so these are not necessary and sufficient conditions. To close this gap is left for future research.

Theorem 5**.**

Under Assumption 1,

(a) any constant withdrawal process $W(t)=w<1-e^{-c-g}$ is sustainable;

(b) any withdrawal process with asymptotic rate $w>c+g$ is not sustainable.

For example, if we take 5-year earnings as fundamentals, then $c+g=6.4\%$ and therefore $1-e^{-c-g}=1-e^{-0.064}=0.062$. Thus any withdrawal process with constant proportional rate less than $6.2\%$ in sustainable. For example, the classic 4% withdrawal rule (withdrawing 4% of current wealth) is sustainable.

4.1. Withdrawal processes depending on earnings growth

One would like explicit formulas for wealth distribution for simple withdrawal processes, for example $W(t)\equiv w$. However, we need to model real earnings growth time series, but we do not impose any assumptions on it. Thus we cannot find such explicit formulas. However, we can create a special withdrawal process dependent on $G(t)$ with wealth process independent of $G(t)$.

Assumption 5**.**

There exists a constant $u>0$ such that $G(t)>-u$ almost surely for all $t$.

Under Assumption 5, we can define the withdrawal process

[TABLE]

Lemma 2**.**

Under Assumption 5, the process (18) is sustainable if and only if $c>u$.

For the real market data, $u=\max(G(1),\ldots,G(152))=0.36>c=0.045$. This assumption 5 does not allow us to construct a sustainable withdrawal process.

5. Background & Discussion

5.1. Sources of stock market returns

The long-run total returns of a stock portfolio, or an individual stock, or the entire stock market, are composed of three parts: dividend yield (dividends paid last year divided by the current price of a stock), earnings growth, and changes in the P/E ratio, or price-to-earnings ratio. For the market as a whole (measured by the Standard & Poor 500 or any other index), the P/E ratio and is computed as the sum of P/E ratios of individual stocks, weighted by market capitalizations (total market value) of stocks. or, equivalently, the ratio of the total market value of all these stocks to their total earnings over the last year. To quote the book [10, pp.324–325]:

Very long-run returns from common stocks are driven by two critical factors: the dividend yield at the time of purchase, and the future growth rate of earnings and dividends. In principle, for the buyer who holds his or her stock forever, a share of common stock is worth the present, or discounted value of its stream of future dividends. Recall that this discounting reflects the fact that a dollar received tomorrow is worth more than a dollar received today. […] The discounted value of this stream of dividends (or funds returned to shareholders through stock buybacks) can be shown to produce a very simple formula for the long-run total return for either an individual stock or the market as a whole: Long-run equity return = initial dividend yield + growth rate. […] Over shorter periods, such as a year or even several years, a third factor is critical in determining returns. This factor is the change in valuation relationships — specifically, the change in the price-dividend or price-earnings multiple.

In other words, dividend yield and earnings growth produce fundamental return, driven by fundamentals (earnings and dividends), while changes in the P/E ratio produce speculative return, driven by market emotions. A classic result in financial theory is that if a P/E ratio of the whole market (measured by S&P or any other comprehensive index) is high relative to the long-term average (which is approximately 16), then the market is overpriced and will deliver low returns in the near future.

The difference between total returns and earnings growth fluctuates as AR(1) around its mean value, which is approximately 4–5%. Consider a stagnant economy (with no population growth and no scientific-technological progress and thus no earnings growth). Then the total return fluctuates around 4–5%. In his classic book, Thomas Piketty observes, see [15, p.206]:

…traditional rate of conversion from capital to rent in the 18th and 19th centuries, for the most common and least risky forms of capital (typically land and public debt), was generally on the order of 5%/year.

In these agrarian societies, economic growth and therefore earnings growth was almost 0%. To quote Piketty from [15, p.353]:

Economic growth was virtually nil throughout much of human history: cobining demographic and economic growth, we can say that the annual growth rate from antiquity to the 17th century never exceeded 0.1–0.2%/year. Despite the many historical uncertainties, there is no doubt that the rate of return on capital was always considerably greater than this: the central value observed over the long run is 4–5%/year. In particular, this was the return on land in most traditional agrarian societies.

Thus 4–5%/year can be thought of as pure return on capital other than growth. The new valuation measure shows stability of this pure return fluctuating around this 4–5% level. Only earnings growth is left as exogenous. Two other components: dividend yield and valuation changes are explained as AR(1). The new measure describes two out of three components of total returns.

5.2. Dividends and buybacks

Continuing the quote on [10, pp.325–326]:

Many analysts question whether dividends are as relevant now as they were in the past. They argue that firms increasingly prefer distributing their growing earnings to stockholders through stock repurchases rather than dividend payments. Two reasons are offered for such behavior — one serves shareholders and the other management.

Companies can distribute earnings to shareholders using buybacks instead of payouts, or reinvest them into business, which raises future earnings and dividends. Whether the payout ratio (fraction of earnings paid out as dividends) is an important indicator is a hotly debated topic, starting from the classic article [12], which gives a negative answer to this question. To capture their argument informally: Lower dividend yield now leads to more earnings reinvested into business and thus raise future earnings and dividend growth. See the survey [2] for a comprehensive review of literature defending each side of this controversy. Recently, corporate payout and buyback policy was incorporated into the classic P/E research by Robert Shiller, see [5, 9]. Thus we can view annual total real return minus annual real earnings growth as implied dividend yield. Arguably, this implied dividend yield is a more comprehensive measure than actual dividend yield.

In the classic Shiller CAPE, we compare only prices and earnings without regard to dividends. For the new valuation measure, we compare total returns with earnings growth only, and dividend yield is featured indirectly, through detrending.

Recall that out of three return components: earnings growth, dividend yield, and changes in the P/E ratio, we labeled the first two fundamental, and the third one speculative. Now we merge the second and the third component in this implied dividend yield. To distinguish between its speculative and fundamental parts, we assume that this implied dividend yield fluctuates around the long-term average. In our research, we find that the intrinsic long-run average for this implied dividend yield is $c=4.5\%$.

However, in the short run it can deviate from this number. If it significantly exceeded this average for the last few years, then the market can be considered overheated and overpriced. Thus the valuation measure is the cumulative sum of all past annual implied dividend yields, minus true long-term average times the number of years elapsed. This detrending will be included into the regression. Thus we regress next year’s implied dividend yield upon last year’s detrended valuation measure.

In each two choices of valuation measure $M(t)$: classic CAPE and $H$; for each choice of averaging window $W$, we get the following decomposition of total returns:

[TABLE]

where $\Gamma$ is earnings growth rate for trailing 5-year averaged real earnings; $M$ is governed by an AR(1), with $\triangle M(t)\equiv M(t)-M(t-1)$; and $\Delta(t)=\ln(1+D(t)/S(t))$ is the logarithmic dividend yield for classic CAPE, and $\Delta(t)=0$ for the two other measures. Recall the three components of total returns: earnings growth, dividend yield, and valuation changes. In the right-hand side of (19), (a) For the classic CAPE, $\triangle M$ is the valuation change, $\Gamma$ is earnings growth, and $\Delta$ is dividend yield. (b) For the measure $H$, $\triangle M$ is the sum of valuation change and dividend yield, and $\Gamma$ is earnings growth.

The original CAPE does not explain stock market returns without an additional term (depending on dividend yield). The two modifications of this model do explain stock market returns as the sum of an AR(1) process and earnings growth.

Before 1920, earnings growth was 1–2% while dividend yield was 5%. After 1970, earnings growth is 4–5%, dividend yield is 2%. Dividend yield is chosen by corporate payout policies. If a company chooses to pay much of its earnings as dividends, then dividend yield is large, but retained earnings (used to pay debt, invest in expansion or R&D, and make stock buybacks) are small; thus earnings growth must be smaller. Conversely, if a company chooses to buy back stock instead of paying dividends, then earnings per share increase faster. Thus actual earnings growth might differ from what one could call fundamental earnings growth, which is growth with non-changing dividend payout/stock buyback policy. Similarly, actual dividend yield might not be representative of fundamental dividend yield. Recall the Modigliani-Miller theorem that states indifference of total returns to dividend policy.

5.3. Comparison CAPE vs new measure

As of January 2023, the CAPE shows the market is overpriced, but the current valuation measure is lower than its long-term average. In Figure 7, we plot these two valuation measures: log CAPE and $H$. Both are centralized by subtracting their historical averages. We can see that these two measures sharply diverge after the start of the 21st century. This can be attributed to recent reluctance of companies to pay dividends. See Figure 8, showing the payout ratio: Fraction of earnings paid out as dividends. We take 5-year trailing averages for earnings and dividends to smooth out annual fluctuations. One can clearly see that after 2000, the payout ratio was much lower than the historical average. This change was rather recent (last 20 years). If this aversion to dividends persists for another century, then CAPE might become elevated and no longer representative of the market conditions. Then the new valuation measure will have stronger predictive power of future returns.

How to explain these variations in the payout ratio? Recent reluctance to pay dividends is partly a consequence of the proliferation of stock options given to executives as part of their compensation package. The latter thus have an incentive to boost the stock price, even at the expense of dividends. Another reason is taxation: Capital gains (increases in stock prices) are often taxed less than dividends.

During the World War II and the generation after that (1940–1970), in contrast to the new millennium, the new measure is much higher than the log CAPE during 1940–1970. Figure 8 explains this as well: Right before this era, during the Great Recession of the 1930s, payout ratio was unusually high compared to the historical average. In fact, for years 1930–1933 annual dividends paid equaled or exceeded earnings, see Figure 2. In 1934–1939 this ratio dropped but was still greater than 50%. This strange phenomenon can be explained as follows: During the economic and financial collapse of the 1930s, earnings plummeted. Dividends decreased as well, but not by much. In times of crisis and recession, companies are reluctant to cut dividends, since this hurts investor confidence. Thus it can happen that dividends temporarily exceed earnings. (The same happened in 2008.)

This anomalously high payout ratio in the 1930s shifted the new valuation measure upward compared to the log CAPE. It stayed this way for three other decades before decreasing below 50% and staying this way for almost all years since then. Only then the log CAPE caught up with the new valuation measure and then substantially exceeded it.

5.4. Adjustment of CAPE for dividends

In the articles [5, 9] we get the modification of CAPE version which Shiller called total returns – cyclically adjusted price-earnings ratio, or TR-CAPE; see Figure 4. This is computed as follows.

Investing 1 at the beginning of data in 1871 and reinvesting dividends, compute total wealth for each year. This wealth grows much faster than the index, as seen in Figure [1](#S1.F1). Divide this wealth by the index level (adjusted so that 1871 = 1) and get the correction coefficient which measures, loosely speaking, wealth produced only by dividends, not by capital gains. We multiply each year’s earnings by this correction coefficient. This gives us modified earnings. Average them over the previous 5 years, and divide the wealth by these trailing 5-year modified earnings. We put wealth, not index level, in the numerator, to adjust it for dividends (or, equivalently, for total return) as well.

Recall our split of market returns into three components: two fundamental (earnings growth and dividend yield) and one speculative (change in valuation). Here we united two fundamental components into one (growth of 5-year trailing total return-adjusted earnings). Thus we can study our specuative component as a stock market valuation measure. This TR-CAPE measure theoretically seems better than the original Shiller CAPE. But, in practice, these measures are very close. The measure $H$ is substantially different from Shiller CAPE; the TR-CAPE is not. Thus we shall not study this modified measure, since it is not an improvement over the classic 5-year CAPE.

5.5. Future research

It would be interesting to model earnings growth. This will include economic forecasts, since corporate earnings are closely tied to overall economy and tax policy. In addition, a more detailed investigation for withdrawal rates is needed. Alternatively, one can replicate this research for individual stocks and stock portfolios, rather than the overall stock market index. Finally, we can include interest rates in our modeling (short-term Treasury bills, long-term Treasury bonds, and long-term AAA corporate rates), which are also available from 1871 onward. We can use these bond yields in our regression models. Low bond yields make them less attractive and push stock prices upward, making CAPE higher. Since now we have historically low yields in the USA, this can explain historically high CAPE ratios. In addition, we can make portfolios of stocks (measured by the S&P index) and bonds (Treasury or AAA corporate bonds).

6. Appendix

6.1. Proof of Lemma 13

Equation (12) follows from (10) and the following computations:

[TABLE]

Equation (13) follows from (8) and (12).

6.2. Proof of Theorem 15

Let us show (14). We can rewrite

[TABLE]

It suffices to show that almost surely,

[TABLE]

The process $H$ is ergodic in the sense of classic ergodic theory; see for example [20, Chapter 6]. The mean of the stationary distribution for $H$ is equal to $c$. By the classic Birkhoff ergodic theorem, see [20, Theorem 6.2.7], we have almost sure convergence

[TABLE]

Obviously, then we have:

[TABLE]

We get (20) immediately from (21) and (22). This completes the proof of  (14). Similarly, (15) follows from (14) and (20).

6.3. Proof of Theorem 2

Define the density of $\mathcal{N}(0,\sigma^{2})$:

[TABLE]

Then we can express $\varepsilon(t)=H(t)-h-b(H(t-1)-h)$ using (4). Therefore, the transition density of $(G,H)$: condition density of $(G(t),H(t))$ at $(g_{1},h_{1})$ given $G(t-1)=g_{0}$ and $H(t-1)=h_{0}$, is given by

[TABLE]

6.4. Proof of Theorem 3

Step 1. The function $\mathbf{p}$ from Assumption 3 is strictly positive for any argument values. The Gaussian density (23) is strictly positive too. Thus, the same is true for the function (24). Therefore, the Markov process $(G,H)$ has the positivity property: For a subset $A\subseteq\mathbb{R}^{2}$ of positive Lebesgue measure, and for any $g_{0},h_{0}\in\mathbb{R}^{2}$,

[TABLE]

Step 2. Solve the recurrent equation (4): $H(t)-h=b^{t-1}\varepsilon(1)+\ldots+\varepsilon(t)+b^{t}(H(0)-h)$. This random variable has the same distribution as

[TABLE]

The following series converges in $L^{p}$:

[TABLE]

Indeed, the $p$-norm of $\varepsilon(s)$ is constant (does not depend on $s$), and $b\in(0,1)$. Therefore, the series (26) converges in $L^{p}$. We apply that the space $L^{p}$ is Banach, that is, every fundamental sequence converges. Thus the sequence $H^{\prime}(t)\to H^{\prime}(\infty)+h$ as $t\to\infty$ in $L^{p}$. Therefore, the distribution of $H(t)$ (the same as that of $H^{\prime}(t)$) converges in law to the distribution of $H^{\prime}(\infty)+h$. In addition, the $p$th moments of $H(t)$ converges to the $p$th moment of $H^{\prime}(\infty)+h$ as $t\to\infty$. Since convergence in $\mathcal{W}_{p}$ is equivalent to weak convergence plus convergence of $p$th moments, the above results implies convergence in $\mathcal{W}_{p}$. Finite $p$th moments of the stationary distribution $\Pi$ are implied by (27) and Fatou’s lemma. We get the marginal distribution of $\Pi$ for $H$ from the observation that $H$ is an AR(1) with Gaussian innovations; see any classic text on time series, for example [4].

Step 3. Finally, let us show uniqueness of the stationary distribution and convergence in total variation. This follows from the two observations. First, as mentioned above, the process $(G,H)$ has the positivity property (25). Second, we have:

[TABLE]

For $G$, this is in Assumption 4; and for $H$, this was shown above, since $H^{\prime}(t)$ converges in $L^{p}$ as $t\to\infty$. From (27), we get that the sequence $(G(t),H(t))$ is tight:

[TABLE]

In [11], the property (28) is called boundedness in probability. This process $(G,H)$ is also irreducible (that is, it does not have two or more disconnected components of the state space $\mathbb{R}^{2}$), since the transition density is strictly positive everywhere. The function $\varphi$ from (23) is contnuous. The function $\mathbf{p}$ from Assumption 3 is continuous in the first two arguments. Therefore, the Markov process $(G,H)$ is a $T$-chain in the sense of [11, p.548]. Applying Corollary (ii) in [11, p.550], we conclude that the Markov process $H$ is positive Harris recurrent in the sense of [11]. Therefore, it has a unique stationary distribution. Applying [11, Theorem 3.4] to Lebesgue measure $\psi$ (since $(G,H)$ is irreducible with respect to this Lebesgue measure), we get: all compact sets are petite. Apply [11, Theorem 5.2(iii)]. Use the aperiodicity ($m=1$ in the notation on the cited theorem), which follows from positivity of the transition density. This completes the proof of Step 3.

6.5. Proof of Theorem 4

We rewrite

[TABLE]

Recall Assumption 2. By the Slutsky theorem, see for example [7, Theorem 5.5.17], it suffices to show convergence in distribution (or, equivalently, in probability):

[TABLE]

But $H(t)\to H(\infty)\sim\pi_{H}$ weakly converges to the stationary distribution. Applying the Slutsky theorem again, $H(t)/\sqrt{t}\to 0$ in probability. This proves (29).

6.6. Proof of Theorem 5

(a) We can rewrite

[TABLE]

where $w^{*}=-\ln(1-w)<g+c$. By induction, $V_{W}(t)=\exp\left[Q(1)+\ldots+Q(t)-tw^{*}\right]$. We can represent this exponent as

[TABLE]

Using an earlier result (20), we get:

[TABLE]

Thus $V_{W}(t)\to\infty$ a.s. as $t\to\infty$.

(b) Similarly, we can represent

[TABLE]

Note that $-\ln(1-w)\geq w$ for $w\in(0,1)$. Thus

[TABLE]

Dividing by $t$ and letting $t\to\infty$, we have a.s. convergence of the right-hand side:

[TABLE]

Thus $\varlimsup\limits_{t\to\infty}V_{W}(t)\leq 1$, and this withdrawal rate is not sustainable.

6.7. Proof of Lemma 2

The corresponding wealth process satisfies the recurrent relation:

[TABLE]

Applying this recurrent relation multiple times, we get:

[TABLE]

Combine (30) with an earlier result (20), we get the following:

(a) If $c>u$, we get: $H(T)+(c-u)T\to+\infty$ as $T\to\infty$, and therefore $V(t)\to+\infty$. The withdrawal process is sustainable.

(b) If $c<u$, then $H(T)+(c-u)T\to-\infty$ as $T\to\infty$, therefore $V(T)\to 0$, and the withdrawal process is not sustainable.

(c) Finally, if $c=u$, then the autoregressive process $H(t)$ does not have an almost sure limit, so the same is true for $V(t)$. The withdrawal process is not sustainable.