Mining Half a Billion Topical Experts Across Multiple Social Networks

TL;DR

This study presents a large-scale, multi-source approach to identify and rank over 650 million topical experts across social networks and web data, demonstrating effective features and a scalable system.

Contribution

It combines data from four major social networks, Wikipedia, and web metadata to develop a comprehensive expertise ranking model at an unprecedented scale.

Findings

Features from Twitter Lists, Wikipedia, and webpages are strong indicators of expertise.

The ranking model effectively identifies experts across 9,000 domains.

The system ranks over 650 million experts daily with validated accuracy.

Abstract

Mining topical experts on social media is a problem that has gained significant attention due to its wide-ranging applications. Here we present the first study that combines data from four major social networks -- Twitter, Facebook, Google+ and LinkedIn, along with the Wikipedia graph and internet webpage text and metadata, to rank topical experts across the global population of users. We perform an in-depth analysis of 37 features derived from various data sources such as message text, user lists, webpages, social graphs and wikipedia. This large-scale study includes more than 12 billion messages over a 90-day sliding window and 58 billion social graph edges. Comparison reveals that features derived from Twitter Lists, Wikipedia, internet webpages and Twitter Followers are especially good indicators of expertise. We train an expertise ranking model using these features on a large ground truth dataset containing almost 90,000 labels. This model is applied within a production system that ranks over 650 million experts in more than 9,000 topical domains on a daily basis. We provide results and examples on the effectiveness of our expert ranking system, along with empirical validation. Finally, we make the topical expertise data available through open REST APIs for wider use.