News-Based Group Modeling and Forecasting

TL;DR

This paper analyzes news group dynamics using NLP-derived data, revealing fame distribution patterns and developing models to forecast future entity fame and popularity within various news groups.

Contribution

It introduces a large-scale NLP-based framework for modeling and forecasting news group fame distributions, including a practical HMM-based news generation model.

Findings

Fame distributions follow log-normal and power-law patterns.

Future fame distribution exhibits a power-law tail.

Forecasting methods can predict future entity fame and probability of becoming very famous.

Abstract

In this paper, we study news group modeling and forecasting methods using quantitative data generated by our large-scale natural language processing (NLP) text analysis system. A news group is a set of news entities, like top U.S. cities, governors, senators, golfers, or movie actors. Our fame distribution analysis of news groups shows that log-normal and power-law distributions generally could describe news groups in many aspects. We use several real news groups including cities, politicians, and CS professors, to evaluate our news group models in terms of time series data distribution analysis, group-fame probability analysis, and fame-changing analysis over long time. We also build a practical news generation model using a HMM (Hidden Markov Model) based approach. Most importantly, our analysis shows the future entity fame distribution has a power-law tail. That is, only a small number of news entities in a group could become famous in the future. Based on these analysis we are able to answer some interesting forecasting problems - for example, what is the future average fame (or maximum fame) of a specific news group? And what is the probability that some news entity become very famous within a certain future time range? We also give concrete examples to illustrate our forecasting approaches.