Nowcasting Stock Implied Volatility with Twitter

TL;DR

This paper demonstrates that Twitter-derived sentiment and attention features improve the prediction of next-day stock implied volatility using random forests, with effectiveness varying across sectors and market regimes.

Contribution

It introduces a novel approach combining Twitter sentiment with machine learning to predict implied volatility and analyzes sector and regime-specific performance.

Findings

Twitter features enhance prediction accuracy.

Prediction performance varies across sectors.

Market regimes influence the effectiveness of the approach.

Abstract

In this study, we predict next-day movements of stock end-of-day implied volatility using random forests. Through an ablation study, we examine the usefulness of different sources of predictors and expose the value of attention and sentiment features extracted from Twitter. We study the approach on a stock universe comprised of the 165 most liquid US stocks diversified across the 11 traditional market sectors using a sizeable out-of-sample period spanning over six years. In doing so, we uncover that stocks in certain sectors, such as Consumer Discretionary, Technology, Real Estate, and Utilities are easier to predict than others. Further analysis shows that possible reasons for these discrepancies might be caused by either excess social media attention or low option liquidity. Lastly, we explore how our proposed approach fares throughout time by identifying four underlying market regimes in implied volatility using hidden Markov models. We find that most added value is achieved in regimes associated with lower implied volatility, but optimal regimes vary per market sector.