Scaling of inefficiencies in the U.S. equity markets: Evidence from three market indices and more than 2900 securities

TL;DR

This study analyzes quote dislocations in U.S. equity markets using comprehensive data, revealing significant inefficiencies, their temporal patterns, and potential costs, especially driven by S&P 500 constituents.

Contribution

It provides the first large-scale quantification of quote dislocations and their impact on trading costs in the U.S. equity markets.

Findings

Over 3.1 billion dislocation segments in 2016

Dislocations are more frequent near market open and close

Estimated opportunity costs around $2 billion USD

Abstract

Using the most comprehensive, commercially-available dataset of trading activity in U.S. equity markets, we catalog and analyze quote dislocations between the SIP National Best Bid and Offer (NBBO) and a synthetic BBO constructed from direct feeds. We observe a total of over 3.1 billion dislocation segments in the Russell 3000 during trading in 2016, roughly 525 per second of trading. However, these dislocations do not occur uniformly throughout the trading day. We identify a characteristic structure that features more dislocations near the open and close. Additionally, around 23% of observed trades executed during dislocations. These trades may have been impacted by stale information, leading to estimated opportunity costs on the order of $ 2 billion USD. A subset of the constituents of the S&P 500 index experience the greatest amount of opportunity cost and appear to drive inefficiencies in other stocks. These results quantify impacts of the physical structure of the U.S. National Market System.

Tables

	coef	std err	z	P$>$$|$z$|$	[0.025	0.975]
Intercept	7.8666	0.802	9.811	0.000	6.295	9.438
l_MarketCap	-0.0738	0.149	-0.496	0.620	-0.365	0.218
l_total_trades	-4.1661	0.432	-9.638	0.000	-5.013	-3.319
l_differing_trades	3.0804	0.338	9.103	0.000	2.417	3.744
l_MarketCap ** 2	0.0067	0.008	0.837	0.402	-0.009	0.022
l_total_trades ** 2	0.3385	0.038	8.936	0.000	0.264	0.413
l_differing_trades ** 2	-0.2042	0.034	-6.002	0.000	-0.271	-0.138

Equations

$$⟨f_A⟩=\frac{1}{N_A}\sum _{\begin{array}{c}1\le n\le N_A\\ f\text{ has condition }A\end{array}}{f}_n.$$

Full text

{classcode}

D50, D82, G14

1 Introduction

Securities markets utilize auction mechanisms to facilitate the valuation and trade of assets (de la Vega, 1688; Bachelier, 1900; Knight, 1921; Fama, 1965). Implementation details of these markets, including the auction mechanism, computing and communication infrastructure, as well as information dissemination policies, impact their informational and economic efficiency (Akerlof, 1970; Easley and O’hara, 1987). These market microstructure factors have been increasingly considered in recent analyses of market efficiency (Wissner-Gross and Freer, 2010; Ding et al., 2014; Mackintosh, 2014; Adrian, 2016), and we contribute to this body of work.

1.1 Modern U.S. market

We investigate a broad subset of the equities traded in the U.S. National Market System (NMS), a network of stock exchanges located in the U.S., since it is the proverbial center of the world equity markets. In particular, we focus on constituents of the Russell 3000 Index, which is compiled by FTSE International Ltd. and contains roughly 3000 of the largest equities traded on the NMS. The selected sample represents the vast majority of the equities traded in the U.S. and can serve as a nearly comprehensive cross-section of publicly traded equities from which the observation and assessment of microstructure quantities can be made.

Using the most comprehensive dataset of NMS messages commercially available, we enumerate and describe dislocation segments (DS) and realized opportunity costs (ROC), defined in Section 5, in Russell 3000 securities and a selection of exchange-traded funds (ETFs) during calendar year 2016. Extending Tivnan et al. (2020), we observe over $3\times 10^{9}$ DSs in the National Market System during the period of study. Consistent with Tivnan et al. (2020), we define a dislocation as the market state when the National Best Bid and Offer (NBBO) and a synthetic, Direct Best Bid and Offer (DBBO) simultaneously display different prices. Any trade that executes during a dislocation is called a differing trade. Row 8 of Table 1.1 shows that 23.71% of all trades were differing trades. Differing trades may have been influenced by stale quote information, so we used them to calculate ROC. However, some trades may have been executed in this period intentionally, so we only include differing trades that executed at either of the two NBBO quotes. This results in a conservative estimate of total ROC, $2,051,916,739.66 across the Russell 3000 in 2016, as depicted in Row 1 of Table 1.1. Table 1.1 depicts the total number of dislocations that we observed in 2016, delineating those that persisted for a minimum duration and those greater than the minimum magnitude (i.e., tick size). Despite recent technological upgrades to market infrastructure, the chief economist at Nasdaq confirms that latencies of 500 microseconds remain material in 2020 for the execution of latency arbitrage strategies Mackintosh (2020).

1.2 Scaling in finance

Mandelbrot (1997, 2013) was one of the first to characterize the scaling properties of price returns in modern markets. The scaling of returns was later revisited by  Stanley and Plerou (2001),  Cont (2001), as well as  Patzelt and Bouchaud (2018). Beyond returns in price time series, additional financial variables have been found to display scaling properties. Market indices and foreign exchange rates (Di Matteo, 2007) as well as share volume and number of trades (Stanley et al., 2008) adhere to scaling properties.

Leveraging the large number of securities under study and the broad range of market capitalization (MC) covered, we examine scaling relationships between DSs, ROC, and MC. DSs occur in equities of all sizes. While DS are more frequent in equities with larger MC, the distributions of their qualities, such as their magnitude and duration, are more extreme among equities with smaller MC. We find a strong positive correlation between MC and ROC, show in Figure 1. A similar relationship is seen between MC-total trades and MC-differing trades in Figure 6. The majority of ROC is generated by equities in the S&P 500 that are not also in the Dow (termed the SPexDow). The SPexDow also Granger-causes ROC in other mutually-exclusive market categories (Dow 30 and Russell 3000 less the S&P 500, or RexSP), pointing to its centrality in the U.S. equities market.

In the following sections, we first provide a brief overview of the U.S. National Market System. We then detail our data, the available and used fields, and summarize the equities studied. After describing statistics of DSs, including distributions of start times and durations, we move to analysis of ROC, providing summary statistics, comparisons across mutually-exclusive market categories, and correlation along with Granger-causality analyses. We close with a brief exploration of exchange-traded funds (ETFs), a discussion of results, and possibilities for future work.

1.3 Empirical Studies of Modern U.S. Markets

In a recent report to its government oversight committee, the U.S. Securities and Exchange Commission (SEC) offered the following characterization of the prevailing literature which relates to our study: “It is unsurprising that academic studies generally are narrowly focused, as the amount of data, computing power and sophistication necessary to engage in broader study are daunting and costly, and relevant data may not be widely available or easily accessible.” (U.S. Securities and Exchange Commission, 2020b, p. 45). Given these constraints, we are aware of only two other recent studies which also used comprehensive, market data to analyze the modern U.S. market.

In the first study,  Wah (2016) calculated the potential opportunities for latency arbitrage on the S&P 500 in 2014 using data from the SEC’s MIDAS platform (U.S. Securities and Exchange Commission, 2013a). Using similar data to that for our study, Wah identified price discrepancies that could serve as latency arbitrage opportunities. She then examined the potential profit to be made by an infinitely fast arbitrageur taking advantage of these opportunities. Wah estimates that this idealized arbitrageur could have captured $3.0B USD from latency arbitrage in 2014, which is similar to our conservative calculations of approximately$2.1B USD in ROC from actual trades in 2016.

The second study was  Aquilina et al. (2020), which used message data from 2015 to quantify aspects of latency arbitrage in global equity markets. The authors note the frequent yet fleeting occurrence of latency arbitrage opportunities and estimate profits from latency arbitrage in 2018 at $4.8B USD globally, including$2.8B USD in the U.S. equity market.

Both the Wah and Aquilina et al. studies relied on affiliations with regulatory agencies and their respective data. This reliance on regulatory data supports the SEC observation that “relevant data may not be widely available or easily accessible.”

2 Market Overview

The U.S. equities market, known as the National Market System (NMS), is composed of 13 National Securities Exchanges. Each exchange contributes to price discovery through the interactions of market participants, mediated by an auction mechanism. Another core component of the NMS is a collection of approximately 40 alternative trading systems (ATSs) (Tuttle, 2013), also known as dark pools. ATSs provide limited pre-trade transparency, which can allow market participants to reduce the market impact of their trades, but have limited participation in price discovery as a result. Each exchange and ATS accumulates orders whose execution conditions have not been met in an order book. Resting orders are matched with incoming marketable orders based on a priority mechanism, commonly price-time priority (Bessembinder, 2001). Traders often have access to a variety of order types that allow them to tailor how they interact with the market (Nasdaq, 2017; New York Stock Exchange, 2020; Chicago Board Options Exchange, 2020; Exchange, 2020). The top of the book at each exchange, the resting bid with the highest price and the resting offer with the lowest price, is called the best bid and offer (BBO). BBOs from across the NMS are aggregated by one of the Securities Information Processors (SIP) to form the national best bid and offer (NBBO) (The Consolidated Tape Association, 2020; The UTP Plan, 2020). Under Regulation National Market System (Reg. NMS), trades must execute at a price that is no worse than the NBBO, though exceptions exist (e.g. intermarket sweep orders) (U.S. Securities and Exchange Commission, 2005).

Market participants in the NMS have several options of data products to fuel their trading decisions. In addition to the dissemination of the NBBO, each SIP provides data feeds containing all quotes and trades that occur in their managed securities. Information feeds offered by each exchange, referred to as direct feeds, can provide similar information with lower latency than the SIP data feeds. Direct feeds can also provide additional data, such as the resting volume at all price points, commonly called depth-of-book information. Information asymmetries between data products lead to DSs, which can impact trading decisions and outcomes.

The NMS is regulated by the U.S. Securities and Exchange Commission (SEC), a federal agency, and self-regulated by the Financial Industry Regulatory Authority (FINRA), a professional organization. FINRA polices its members and ensures they adhere to SEC rules and other professional guidelines, while SEC designs, implements, and enforces rules that are intended to promote market stability and economic efficiency. The physical structure of the NMS, in conjunction with the existence and usage of multiple distinct information feeds, leads to the creation of DSs and associated ROC. On Dow 30 equities, over 120 million DSs and over $160M USD in ROC were cataloged during calendar year 2016 (Tivnan et al., 2020).

Our calculations provide a conservative estimate of ROC from actual trades in the U.S. equity markets in 2016. Therefore, we identify some relevant literature on trade execution (U.S. Securities and Exchange Commission, 2013b); namely, where and when trades occur. First, trading is not instantaneous. Delays, or latencies, exist throughout the NMS. Second, not all trading activity occurs at a national exchange or an ATS. Instead of routing an order to one of these market venues, a broker may execute the order against the broker’s own inventory of that stock. This process of retaining customers’ orders internal to the brokerage is called “internalization” (U.S. Securities and Exchange Commission, 2000). In addition to matching customers’ orders against the broker’s inventory of a particular stock, internalization also includes instances when a broker may route customers’ orders to a market-maker under a Payment for Order Flow (PFOF) agreement. Even without charging commissions for trades, brokers may generate revenue from executing trades via PFOF (U.S. Securities and Exchange Commission, 2007). To mitigate potential conflicts of interest, each broker is required to ensure that its customers’ orders execute against best prices, as determined by the NBBO.

Trade execution problems may still arise from PFOF. In a public statement announcing its fine against a prominent market-maker, the SEC noted the use of algorithms which were used to avoid paying best prices on internalized orders. Per the SEC, “these algorithms were triggered when they identified differences in the best prices on market feeds, comparing the SIP feeds to the direct feeds from exchanges” (U.S. Securities and Exchange Commission, 2017). The reader will note that this market state, what the SEC has identified as “differences in best prices on market feeds,” is the very same state that we have defined here as a market dislocation.

PFOF remains a controversial practice. More recently, another market-maker settled allegations that it did not ensure best prices for the internalization of its customers’ orders (Michaels, 2019).

We found references to internalization and PFOF dating back to 1994, when annual revenues from PFOF exceeded 500M USD across all U.S. brokers (Power, [1994](#bib.bib37)). Some studies identified the potential for conflicts of interest from PFOF, but claimed that these conflicts could be mitigated by the adoption of minimum tick sizes of a penny (i.e., decimalization) (Chordia and Subrahmanyam, [1995](#bib.bib12); Easley *et al.*, [1996](#bib.bib17)). Though the SEC adopted decimalization in 2000 (U.S. Securities and Exchange Commission, [2020a](#bib.bib54)), PFOF remains a lucrative practice. In the first half of 2020, four brokers in the U.S. generated more than 1B USD in revenue from PFOF (Wursthorn and Choi, 2020).

This brief overview of the U.S. equities market only provides context for the following sections and is far from complete. We refer the reader to Tivnan et al. (2020) for a more complete discussion.

3 Indices

Many of our results are centered around the components of three of the most popular equity indices: Dow Jones Industrial Average, S&P 500, and the Russell 3000. Indices measure the performance of a bucket of securities. The choice of the underlying securities is often to be representative of a market segment. Indices may not be directly purchased in the same way as an equity, but may be tracked by Exchange Traded Funds (ETFs) and mutual funds.

The Dow Jones Industrial Average, from here on referred to as the Dow, is a price weighted index that aims to provide an overview of the U.S. economy (S&P Dow Jones Indices, 2020a). The Dow consists of thirty S&P 500 constituents, covering all industries except for utilities and transportation.

The S&P 500 is a market capitalization weighted index of 500 large US based companies referred to by it’s creators as “the gauge of the market economy”. The index is considered by many to be representative of the US stock market as a whole and is a primary holding among passive investors. To be included in the index, as of 2016, a company must meet the following criteria (S&P Dow Jones Indices, 2020b).

• •

Be a U.S. Company

• •

Have a market capitalization greater than $5.3 billion

• •

Be highly liquid

• •

Have a public float of at least 50% of outstanding shares

• •

Had positive earnings in the most recent quarter

• •

The sum of the last four consecutive quarterly earnings must be positive

• •

Be listed on a major exchange

Meeting these criteria does not guarantee inclusion, and failing to uphold these standards does not necessarily result in immediate expulsion from the index. S&P 500 constituents are chosen by S&P Global, and the index is updated regularly, though not on any fixed schedule.

The Russell indexes are passively constructed (no human in the loop) based on a transparent set of rules including (FTSE Russell, 2020b):

• •

Be a U.S. Company

• •

Be listed on a major exchange

• •

Have a share price $\geq$ $1

• •

Have a market capitalization $\geq$ $30M

• •

Have a public float $\geq$ 5%

The Russell 3000 consists of the largest 3000 firms by market capitalization meeting the above criteria, or the entire eligible set, whichever is smaller. The index undergoes an annual reconstruction in June and is augmented quarterly with the addition of Initial Public Offerings (IPO). This methodology results in the Russell 3000 being a strict superset of the S&P 500.

For our analysis we focus on constituents of these indices, rather the index itself. Thus, differing weighting methodologies used by these indices have no effect on our analysis. We also note that some companies have multiple common stocks, one for each share class, and that each index handles the inclusion of multiple share classes differently.

4 Data

We use a dataset comprised of every quote and trade message that was disseminated on one of the SIP or direct feeds during the period of study. This dataset features comprehensive coverage of the stocks under study, is collected from a single location (Carteret, NJ), and is time stamped upon arrival, thus limiting clock synchronization issues. Thesys Technologies collected and curated this data (Thesys Technologies, 2020), and also provided data for the SEC’s MIDAS (U.S. Securities and Exchange Commission, 2013a) at the time of collection. Index membership, Global Industry Classification Standard (GICS) sector classification, and MC data were obtained from a Bloomberg Terminal.

The indices we consider are subject to frequent changes in membership. To simplify our analysis we consider the Dow 30 and S&P 500 as they stood on Jan. 1, 2016. For the Russell 3000 we consider the constituents as listed in the June 2016 construction, excluding those that were not publicly traded on Jan. 1, 2016. Constituents of the indices under study were curated to only include companies that survived as a publicly traded entity on a national exchange for the entire calendar year of 2016. Companies that were delisted for any reason (e.g. bankruptcy or buyout) were excluded, in addition to those who were acquired by an out-of-study firm. Mergers between in-sample companies did not result in exclusion. Curating the stocks under study in this way allows us to avoid issues caused by IPOs and delistings.

Many companies in our dataset changed their ticker symbol over the course of the calendar year and thus appear as a different entity in the data. To study a company over a long time period it is necessary to know all tickers it traded under and when the ticker changes occurred. There is no consolidated public record of these ticker changes, so we tracked them via an extensive review of press releases. These ticker changes were then validated by observing changes in trading activity in the old and new ticker on the date of the change using the Thesys data archive.

This curation reduced the Russell 3000 from 3005 stocks to 2903, the S&P 500 from 500 stocks to 472, and did not impact the 30 members of the Dow. We denote the curated version of an existing index by appending a prime to the respective base index (e.g. Dow 30 $\rightarrow$ Dow 30*′). We then construct two additional stock groups, RexSP and SPexDow, by taking the appropriate set difference, e.g. SPexDow = S&P 500′* - Dow 30*′*. Finally, all companies in our dataset were classified by their MC as it stood in the beginning of Q4 2016 using the classes defined in Table 4. Our dataset covers approximately 98% of all publicly traded U.S. equities by MC (FTSE Russell, 2020a). Tables 4 - 4 provide summary statistics and distribution of these equities across GICS sector, MC, and market category, for several indices.

5 Methods

Our work investigates the occurrence of DSs and ROC arising from quote discrepancies between the SIP and direct feeds. Similar concepts have been discussed in empirical market microstructure literature (Arnuk and Saluzzi, 2009; Jarrow and Protter, 2012; Hasbrouck and Saar, 2013; Ding et al., 2014; O’Hara, 2015), though formal definitions vary. We follow the definitions described in  Tivnan et al. (2020), which are briefly reviewed below.

Suppose that there exist two market data feeds, $F_{1}$ and $F_{2}$, each displaying quotes for a single asset. Quotes have the form $q_{i}(t)=(b_{i}(t),m_{i}(t),o_{i}(t),n_{i}(t))$, where $i\in{1,2}$, $b_{i}(t)$ is the bid price at time $t$, $o_{i}(t)$ is the offer priceat time $t$, $m_{i}(t)$ and $n_{i}(t)$ are the number of shares associated with the bid and offer at time $t$ respectively. We observe these feeds from a single, fixed location in Carteret, NJ. A dislocation between these sources of data occurs when the prices of the quotes differ, e.g. $b_{1}(t)\neq b_{2}(t)$ or $o_{1}(t)\neq o_{2}(t)$. A DS occurs when the quotes differ and the relationship between the quoted prices remains constant, e.g. $b_{1}(t)<b_{2}(t)$ or $b_{1}(t)>b_{2}(t)$. Figure 2 walks through an example DS occurring on a subset of the NMS using estimates of message transit and processing time for each leg of the journey. In our example a DS starts when a message regarding a quote change in Mahwah reaches our observer in Carteret via a direct feed and ends when the same message arrives via the SIP $92\mu s$ later. In this single example we see three factors that either alone, or in combination, may cause DSs; differences in processing time, transfer speed, and route (SIP messages require an additional leg). In this example the dislocation was triggered by a single top of book change at NYSE. However, dislocations can occur due to sequences of events occurring across multiple exchanges and SIP processors. Recall that by definition a DS requires two feeds. TAQ data contains only the quotes resulting in a NBBO change as well as all trades. In contrast our dataset contains all quotes sent along the direct feeds as well as all SIP updates. Thus, we can observe events such as our example in Figure 2, an impossibility with TAQ data.

The ROC of using $F_{1}$ instead of $F_{2}$ is calculated by combining quote and trade information. Assume that trades take the form $T_{j}=(p_{j},v_{j},t_{j})$, where $p_{j}$ is the execution price, $v_{j}$ is the number of traded shares, and $t_{j}$ is the execution time. If a trade executes at one of the currently quoted prices, e.g. $b_{1}(t_{j})$, then the ROC is given by $(b_{2}(t_{j})-b_{1}(t_{j}))*v_{j}$. If the trade executes on the opposite side of the book, e.g. $o_{2}(t_{j})$, then the ROC is given by $(o_{1}(t_{j})-o_{2}(t_{j}))*v_{j}$. This allows for a consistent interpretation of the values, where a positive value indicates that $F_{2}$ displayed a better price for the active trader (higher bid or lower offer) than $F_{1}$. The total ROC over an interval $[S,\ E]$ is obtained by taking the sum of ROC values from all trades that occurred in that interval.

We first compute summary statistics and qualitative descriptions of the distributions of DSs and ROC. Additionally, we leverage the large sample of equities to conduct a cross-sectional study of the effect of company “size” on these microstructure quantities. We quantify the notion of size of a company by both its MC and its rank in relation to other companies. We also investigate index inclusion effects through the use of disjoint sets of equities and compute aggregate statistics across these sets. Since the S&P 500 is a strict superset of the Dow 30 and the Russell 3000 is a strict superset of the S&P 500, the natural division of the superset of all equities under study is split into three distinct classes: the Dow 30, the S&P 500 excluding the Dow 30 (SPexDOW), and the Russell 3000 excluding the S&P 500 (RexSP). We investigate correlations between these disjoint subsets, and characterize the statistical properties of the time series of DSs and ROC across these disjoint categories. We further explore the relationship between these categories by conducting a Granger causality analysis of aggregated ROC time series (Granger, 1969).

The next section gives results on DSs, including summary statistics and regressions of DSs against MC. We then discuss structure in the intra-day distribution of DS start times and DS duration. Following this, we provide statistics of the ROC across the market as a whole and again within mutually-exclusive market categories. We then explore statistical properties of the ROC time series. We close with an overview of the statistics of ETF DSs and ROC, contrasting these with those of the market as a whole.

6 Results

6.1 Dislocation Segments

DSs can occur when quotes displayed by distinct information feeds differ. We cataloged all DSs occurring in the equities under study and present summary statistics along with qualitative comparisons of their distributions and higher-order moment statistics. Table 8 - 8 display means of summary statistics of DSs for each mutually-exclusive market category under study.

We will use the notation $\left\langle f_{A}\right\rangle$ to denote an average of the quantity $f$ conditioned on the condition $A$. These averages are interpreted as the quantity $f$ conditioned on condition $A$ averaged over all securities and all times of observation; defining the number of instances of the quantity $f$ having condition $A$ as $N_{A}$, we have

[TABLE]

Tables 8 - 8 show that, on average, there were more DSs in Dow 30 securities than in SPexDow or RexSP securities. However, the average maximum magnitude of DSs in the Dow30 is lower than those of the SPexDow, which in turn are lower than those of the RexSP. In particular, actionable DSs (those with duration $>545\mu s$) with magnitude $>$ $0.01 exhibit more extreme behavior in the SPexDow and RexSP than in the Dow. On average, the median maximum magnitude in the Dow 30 among actionable DSs was$\left\langle\text{median max mag}{duration,magnitude}\right\rangle\simeq$0.023$, while in the SPexDow we observed$\left\langle\text{median max mag}{duration,magnitude}\right\rangle\simeq$0.034$and in the RexSP$\left\langle\text{median max mag}_{duration,magnitude}\right\rangle\simeq$0.045$, a roughly one-cent increase in the median maximum magnitude of a DS in each mutually-exclusive market category. Examples of distributions of these quantities are given in Figure 3, where the distributions of the means of minimum magnitude, maximum magnitude, and duration are plotted for the RexSP.

These results provide evidence for the existence of a MC scaling effect in DSs. Securities with larger MC tend to feature higher trading volume and more frequent occurrence of DSs, but these DSs tend to be smaller in magnitude on average. More frequent trading implies a lower probability that prices across differing information feeds will diverge by large magnitudes.

Since DSs are not distributed evenly throughout the day in the Dow 30 (Tivnan et al., 2020), we examine their distribution in the SPexDow and the RexSP as well. Appendix 9 contains figures displaying the distribution of DS start times plotted modulo day and aggregated over the year as well as figures displaying the distribution of DS durations for each mutually exclusive market category. Distributions are plotted both without conditioning, conditioned on duration, as well as conditioned on duration and magnitude.

Distributions of start times display predictable structure. In all market categories, there are large peaks at the very beginning and end of the trading day (circa 9:30 AM and 4:00 PM), along with a noticeable and sudden increase in density around 2:00 PM. The peak in density that occurs at the end of the day is most noticeable when the distribution of start times is not conditioned on DS size. These observations correspond with the results found for the Dow 30 in Tivnan et al. (2020). However, along with these granular observations, there also exists structure on shorter timescales. The distribution exhibits self-similarity on the half-hour timescale, with large peaks every half-hour and decreasing density toward a sudden peak at the next half-hour. There is also structure at the five-minute timescale that is noticeable before the 2:00 PM spike in density but does not appear to be present after the spike. (Future work could statistically test for the presence of this structure and for its persistence across multiple timescales.) The structure on shorter timescales is present in all distributions but, again, is more pronounced in distributions not conditioned on magnitude.

Distributions of DS duration are extremely heavy tailed, so we plot them with a log-transformed horizontal axis. All DS duration distributions exhibit one or more peaks in the range $10^{-4}s\leq\log_{10}\text{duration}\leq 10^{-3}s$, but there is also a distinct and much lower peak in the distribution near approximately one second in length.

S&P 500 Inclusion Effect: Dislocations

As a visual aid to these results, we have included circle plots, as introduced in Tivnan et al. (2020), to demonstrate the non-uniform distribution of DSs that can occur. Figure 4 shows these circle plots for two common stock pairs ((PBI, INCR), (BRK.B, XOM)) on the edges of our indices. The first pair is the smallest common stock in the S&P 500$\prime$ by MC that remained in the S&P 500 for the entire calender year and the closest component by MC in the RexSP, PBI and INCR respectively. The second pair is the only mega cap in the RexSP and the closest component by MC in the S&P 500$\prime$ that remained in the S&P 500 for the entire calender year, BRK.B and XOM respectively. We note that BRK.A is not included in the Russell 3000 (FTSE Russell, 2020b) and that XOM is additionally included in the DOW. These common stock pairs underscore the difference in behavior between constituents of the S&P 500 and those not included in the most worlds most widely tracked equity index.

Figure 4 displays the DSs for the above-mentioned common stocks aggregated over a year (modulo day). We see that DSs appear to be more concentrated for S&P 500 constituents with spikes occurring at the beginning of the trading day and at 2:00 pm. Additionally, DSs for S&P 500 constituents tend to have smaller magnitudes, relative to Russell 3000 constituents. We provide circle plots for many more securities on our webpage (Ring IV, 2020).

6.2 Market capitalization

Further evidence for scaling behavior arises from analysis of MC. Tables 4 and 4 display MC statistics broken down by industry sector and categorical size, e.g., micro-cap, mega-cap, etc. MC is significantly positively correlated with ROC. Tables 10 - 10 display results from ordinary least squares regressions predicting ROC using MC and other predictors. A linear fit predicting $\log_{10}$ ROC from $\log_{10}$ MC, $\log_{10}$ total trades, and $\log_{10}$ differing trades gives $R^{2}\simeq 0.908$. A positive coefficient relates $\log_{10}$ ROC to $\log_{10}$ MC, indicating that higher MC is associated with higher ROC. A similar regression is computed including quadratic terms in $\log_{10}$ MC, which has a significant, but weak, negative association with ROC. Similar relationships hold for both the linear and quadratic models when the dependent variable is instead chosen to be total or differing trades.

Though behavior of ROC as a function of MC is generally similar when equities are stratified by sector, some sectors display lower average levels of ROC, differing trades, or total trades when MC is held constant. Equities classified as being in the financial sector generally have a smaller amount of ROC, while equities classified as being in the energy sector exhibit a higher amount of ROC on average. However, there is no clear general trend linking sectors to MC or to ROC.

6.3 Realized opportunity cost

As expected with an increase in the number of analyzed equities from 30 to more than 2900, the amount of ROC rose substantially from the quantity reported in Tivnan et al. (2020), from $160M to$2.05B USD. ROC clearly displays sublinear scaling with the number of studied equities; we do not observe a thousandfold increase in the amount of ROC with a thousandfold increase in the number of equities. The information advantage afforded traders with access to direct feed information is not uniform; though a vast majority of the ROC ($1.91 B) favored the direct feeds in this way, a non-negligible amount of ROC ($137 M) did favor the SIP feeds. Approximately a quarter (23.71%) of all trades observed occurred during a dislocation. The fraction of “differing traded value”—the nominal market value of all differing trades—was slightly higher (25.25%) than the fraction of all trades that were differing trades. The ratio between these two values (25.25% / 23.71% = 1.0651) shows that the average differing trade moves approximately 6.51% more value than the average trade. This indicates a qualitative shift in trading behavior during dislocations.

Securities in the SPexDow account for a median of 2,006,091 differing trades per day, in contrast to the 309,158 in the Dow 30 or 1,921,121 in the RexSP. The median differing traded value per day in the SPexDow was also the highest among the three categories, totaling approximately $14.07T versus the RexSP’s total of$6.7T and the Dow’s total of $3.27T. ROC per share differed across the three categories, with median ROC per share per day of 1.1¢, 1.5¢, 2.1¢for the Dow, SPexDow, and RexSP respectively. ROC per share tends to increase as MC decreases, with lowest ROC per share occurring in the Dow and highest ROC per share occurring in the RexSP. Median total ROC per day on the Dow amounted to$514.8K, while median total ROC per day on the SPexDow totaled $3.384M and on the RexSP amounted to$3.564M. Summary statistics for distributions of ROC for each mutually-exclusive market category are given in Table 8.

It is interesting to consider the distribution of both total ROC and ROC per share by both equity and mutually-exclusive market category. Figure 7 displays ROC of the top 30 and bottom 30 of all securities under study when ranked by ROC. Included in this figure for comparison is the exchange-traded fund SPY, an ETF that tracks the S&P 500. Selected ETFs are also treated separately in Section 6.4. It is notable that the equity with the largest ROC, Bank of America (BAC), has more than twice the ROC of the equity with the second-largest amount of ROC, Verizon (VZ). Though not an equity and not included in the rest of this study, it is also notable that SPY, one of the most heavily traded securities on the NMS along with BAC, is close to BAC in ROC. Of the top 30 securities with the most ROC, eight of the 30 are Dow 30 equities. Only four out of 30 are RexSP equities, while the other 17 non-ETF securities are SPexDow equities. One may attribute this to MC, though we note the S&P 500 is not the largest 500 U.S. companies 3. In fact, there are 612 RexSP constituents with a MC greater than PBI, a common stock at the bottom of the S&P 500. This includes 67 large and mega cap common stocks. Since the S&P 500 appears to be the primary driver of ROC across all equities (c.f. below), we find the top 30 and bottom 30 S&P 500 securities ranked by ROC, including Dow 30 securities, and plot their ROC in Figure 8. Even in this subset, only 10 of the top 30 equities are Dow 30 securities. However, when the unit of analysis changes to ROC per share, as in Figure 9, we find that RexSP equities fill 27 out of 30 top ranks, which corresponds with the aggregated statistics reported in Table 8. Additionally, we revisit our common stock pairs from 6.1 to take a closer look at common stocks barely inside and outside of the S&P 500. We see that the common stocks in the S&P 500 have a much higher trading volume and ROC then their similarly capitalized counterparts 6.3. To see if this trend holds we expand our set to the ten smallest common stocks that remained in the S&P 500 for all of 2016 and the ten RexSP common stocks with the closest MC. None of the ten RexSP members spent any time in the S&P 500 during 2016. We find the trend holds with members of the S&P 500 having nearly three times the trading activity and ROC than their similarly capitalized counterparts 6.3.

Since there appear to be differences between the (stationary) summary statistics of the mutually-exclusive market categories, it is reasonable that there may be significant differences between the ROC statistics considered as time-dependent stochastic processes and simply considered as random variables decoupled from time. Within each category, the ROC was computed for all equities in that category for each day. Each ROC series is then normalized as $r_{i}\mapsto\frac{r_{i}-\langle r_{i}\rangle}{\sqrt{\text{Var}(r_{i})}}$, which allows direct comparison of the series. Figure 13 displays a quantile-quantile plot of the Dow, SPexDow, and RexSP ROC distributions. The Dow distribution is plotted as linear and the other two distributions are compared with it. It is immediately obvious that the left tails of the SPexDow and RexSP distributions are heavier than that of the Dow; this also appears to be the case for the right tails of the distributions, but there is little sampling in this region and so no conclusion can be drawn. This similarity of the SPexDow and RexSP distributions is also striking; when normalized they appear almost identical.

Figure 14 displays the time-dependent sample paths of ROC sampled at daily resolution. These processes are anti-autocorrelated—they display mean reversion—as evidenced by their detrended fluctuation analysis (DFA) (Peng et al., 1994) exponents of $\alpha_{\text{Dow}}=0.438$, $\alpha_{\text{SPexDow}}=0.242$, and $\alpha_{\text{RexSP}}=0.235$. All series exhibit rare large values from time to time, with the Dow ROC series exhibiting the largest rare values relative to its mean fluctuations and the SPexDow series exhibiting the smallest. We also note that, in accordance with the QQ plot of the time-decoupled distributions above, the DFA exponents of the SPexDow and RexSP—and thus their corresponding dynamical behavior—are closer than they are to the Dow DFA exponent.

A review of the above results points to the SPexDow as being the “dominant” mutually-exclusive market category in some sense: it accounts for a plurality of differing trades, differing traded value, and total ROC, while also having a DFA exponent lower than that of the Dow and close in value to that of the RexSP, meaning that its time-series of ROC is strongly mean-reverting. The amalgamation of these facts can be interpreted as evidence that the SPexDow ROC time series is possibly least likely to be influenced by the other series of ROC. To test this hypothesis, we conduct a number of Granger causality tests on the time series of ROC. Granger causality is the notion that past values of one time series may be useful in predicting current and future values of another time series (Granger, 1969). A maximum lag of 40 days was set and four tests were calculated pairwise between each of the three mutually-exclusive market categories: sum of squared residuals $\chi^{2}$-test, a likelihood ratio test, sum of squared residuals $F$-test, and a Wald test. We consider there to be a significant Granger causality between series when all four tests indicate significant Granger causality at the $p=0.05/N_{\text{lags}}$ confidence level. The correction for multiple comparisons is done using the most conservative estimate, the Bonferroni correction, to minimize the probability of Type I error (Bonferroni, 1936). Figure 5 displays the results of these tests graphically as a directed network. The direction of edges denotes the direction of the Granger-causal relationship between the categories, while the weights on the edges denote the total number of lags for which the relationship was significant. The SPexDow is shown to significantly influence both the Dow and RexSP while not being significantly influenced by either category; this provides strong evidence to support our above hypothesis. We note that the SPY tracks the S&P 500, is one of the most heavily-traded securities, and has the second-highest amount of ROC of the securities under study here. The SPY’s price dynamics and ROC may thus have a material effect on the relationships between the S&P 500’s ROC and those of the other market categories, providing a partial confounding effect to the Granger-causal relationship determined here; there may be a mutually-causal relationship between the real S&P 500 and the ETF that tracks it. The RexSP and Dow have a mutually Granger-causal relationship, with the Dow exerting more influence on the RexSP than the other way around. This finding corresponds with the ranking of categories on a total shares traded per number of equities basis; this is not a surprising result. We also find that the SPexDow exerts far less influence on the RexSP than does the Dow (four total lags for the SPexDow versus 23 total lags for the Dow), a fact for which we do not have a ready explanation.

Providing further evidence for the above hypothesis, we compute Pearson correlations between pairs of mutually exclusive categories for both ROC and ROC per share; these results are displayed in Table 6.3.

ROC correlations are strongest between SPexDow and RexSP ($\rho=0.72$) and SPexDow and Dow ($\rho=0.45$), while the correlation between the RexSP and Dow is lower ($\rho=0.31$). ROC per share correlations are universally lower than those for ROC, but the correlations between SPexDow and RexSP ($\rho=0.41$) and SPexDow and Dow ($\rho=0.10$) are still higher than that between RexSP and Dow ($\rho=-0.01$), which is actually negative.

Figure 10 displays the distributions of daily total ROC in 2016 by mutually-exclusive market category. The panel with linear scaling highlights the extremely heavy-tailed nature of these distributions, while the log scaled panel provides a better comparison between the mutually-exclusive market categories. On average, members of the Dow 30 experience the greatest daily ROC, followed by members of the SPexDow, followed by members of the RexSP. It seems likely that the kurtosis of the theoretical distributions do not exist, implying tail exponent $\gamma<4$ in the distribution $\Pr(X>x)\sim x^{-(\gamma-1)}$. Table 8 displays the skew and kurtosis for each distribution. If we examine the daily ROC per share in a similar manner, which is shown in Figure 11, we observe a reversal of the previous relationship. Members of the Dow 30 have the least daily ROC per share, on average, and members of the RexSP have the most. Though there is a slight trend, more ROC per share in less frequently traded stocks, the distributions of all three groups are nearly centered at 1¢ per share. This corresponds well with our expectations based on the structure of the system and the distribution of DS magnitudes shown in Figure 3.

6.4 ETFs

Exchange traded funds (ETFs) are securities that trade on the NMS and are designed to mimic as closely as possible a particular portfolio of other securities. They are thus governed by the same price discovery mechanism as other securities that trade on the NMS, as opposed to the end-of-day price discovery mechanism to which mutual funds are subjected, but also allow investors to own a portion of potentially many underlying assets (or at least a simulacrum of such), similar to a mutual fund. Here, we briefly remark on the similarities and differences between ETFs designed to track subsets of the market and those subsets of the market themselves. We calculate statistics on the DSs and ROC associated to ETFs from three firms (Vanguard, iShares, Russell) for three indices (S&P 500, Russell 300, Russell 2000), for a total of nine ETFs (SPY, VOO, IVV, THRK, VTHR, IWV, TWOK, VTWO, IWN). The Russell 2000 is comprised of the smallest 2000 firms in the Russell 3000 by MC. The S&P 500 and Russell 3000 were selected as measures of overall market activity while the Russell 2000 was selected to isolate dynamics among ETFs that track smaller equities.

Table 8 summarizes ROC statistics for the ETFs under study. The fraction of differing trades and differing traded value are lower than for any of the indexes as a whole; in fact, the ratio of the fraction of differing traded value to the fraction of differing trades is less than one. Total ROC incurred from trades in ETFs studied here totaled over $38 million in calendar year 2016. This statistic provides some evidence to suggest that ETFs have their own endogenous statistical behavior that differs from the behavior of the assets from which they are derived.

7 Conclusion

In sum, we have demonstrated that the existence of DSs and ROC is not restricted to Dow 30 securities. Furthermore, we have established that these microstructure quantities occur with non-negligible frequency and size; we show that total ROC in Russell 3000 securities was well in excess of 2 billion USD during 2016. While consistent with the two comprehensive studies of the modern U.S. market (Wah, [2016](#bib.bib56); Aquilina *et al.*, [2020](#bib.bib3)), our ROC calculations provide the first empirical evidence explaining how traders might profitably exploit market dislocations, despite paying up to 2.0B USD annually for order flow (Wursthorn and Choi, 2020).

Compounding these results, we provide strong statistical evidence that the S&P 500 excluding Dow 30 securities, to which we refer as the SPexDow, is the primary driver of ROC among the three mutually exclusive categories of equities (Dow 30, SPexDow, and Russell 3000 excluding S&P 500 securities, or the RexSP).

Compounding the above results, we find that structure in the distributions of DS start times and duration persist across the entire Russell 3000, indicating some broader microstructure-based proximate cause of this structure. Distributions of DS duration exhibit a large peak between $10^{-4}$ and $10^{-3}$ seconds (100 microseconds to one millisecond), but also exhibit a second smaller, yet distinct, peak near one second. This separation of timescales in the distribution provide evidence for the existence of at least two distinct proximate causes of DS. Distributions of DS start times display even more intricate structure, with large peaks at the beginning and end of the trading day, self-similarity on the half-hour and ten-minute timescales, and a large spike at 2:00 P..

ROC was highest among SPexDow securities, but ROC per share was highest among RexSP securities, which were also the most lightly-traded securities. All time series of ROC exhibit behavior of anti-autocorrelation, meaning that they are mean-reverting. ROC in the SPexDow Granger-cause ROC in the other market categories, but the converse is not true; while the Dow Granger-causes the RexSP, the RexSP only weakly Granger-causes the Dow and does not have any effect on the SPexDow.

Taken together, these results paint the picture of a NMS the physical structure of which generates effects that are persistent across size of equity and exchange. Amplifying these persistent effects is the apparent central role of the SPexDow; in number of DSs, amount of ROC, spectral properties of ROC time series, and Granger-causal relationships, the story emerges of the SPexDow’s characteristics being generated by largely-endogenous factors and subsequently influencing the characteristics of the Dow and RexSP. Future work could explore in more depth the extent to which microstructure effects arising first in the SPexDow then spread to other mutually exclusive market categories and propagate through time. This work could also explore the evolutionary dynamics of the modern NMS from its birth following the financial crisis of 2007/8 to the present day. The NMS may not have remained static, with a constant number of market centers and a stationary distribution of market agents and trading strategies, but rather may have experienced fluctuations in the number of exchanges, in the regulatory environment, and in strategy profiles of trading agents. Such an analysis could pave the way for better informed modelling efforts and the advancement of market theory.

Acknowledgements

The authors gratefully acknowledge helpful discussions with Anshul Anand, Yosry Barsoum, Lashon Booker, David Bringle, Eric Budish, Peter Carrigan, Gary Comparetto, Peter Sheridan Dodds, Bryanna Dienta, Jordan Feidler, Andre Frank, Bill Gibson, Frank Hatheway, Michelle Herd, Emily Hiner, Chuck Howell, Eric Hunsader, Robert Jackson, Neil Johnson, Matthew Koehler, Blake LeBaron, Connie Lewis, Matthew McMahon, Wade Shen, David Slater, Jonathan Smith, Paul Soldate, Brendan Tivnan, Jason Veneman, Elaine Wah, Sanith Wijesinghe, and Thomas Wilk.

Funding

All opinions and remaining errors are the sole responsibility of the authors and do not reflect the opinions nor perspectives of their affiliated institutions nor that of the funding agencies. D.R.D., C.M.V.O., J.H.R., and B.F.T. were supported by DARPA award #W56KGU-17-C-0010. C.M.D. was supported by NSF grant #1447634. C.M.D. was supported by a gift from Massachusetts Mutual Life Insurance Company. The authors declare no conflicts of interest. The views, opinions and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

8 Tables

9 Figures

10 Statistics