Choosing News Topics to Explain Stock Market Returns

TL;DR

This paper compares methods for selecting news topics to explain stock returns, finding that plain LDA with random search and reinforcement techniques outperform supervised LDA in out-of-sample predictions.

Contribution

It introduces a practical approach using plain LDA with random search and reinforcement to improve topic selection for stock return explanation.

Findings

Plain LDA with random search outperforms supervised LDA.

Reinforcement of effective topic assignments improves performance.

Empirical tests on 90,000 news articles validate the methods.

Abstract

We analyze methods for selecting topics in news articles to explain stock returns. We find, through empirical and theoretical results, that supervised Latent Dirichlet Allocation (sLDA) implemented through Gibbs sampling in a stochastic EM algorithm will often overfit returns to the detriment of the topic model. We obtain better out-of-sample performance through a random search of plain LDA models. A branching procedure that reinforces effective topic assignments often performs best. We test methods on an archive of over 90,000 news articles about S&P 500 firms.