News Meets Microblog: Hashtag Annotation via Retriever-Generator

TL;DR

This paper introduces a novel method for hashtag annotation in microblogs by leveraging news articles with a Retriever-Generator framework, outperforming previous conversation-based approaches.

Contribution

It proposes using pre-existing news articles instead of conversations to improve hashtag generation, addressing a key limitation of prior methods.

Findings

Outperforms conversation-based methods in hashtag accuracy

Leveraging news articles significantly improves hashtag generation

Demonstrates effectiveness on English Twitter datasets

Abstract

Hashtag annotation for microblog posts has been recently formulated as a sequence generation problem to handle emerging hashtags that are unseen in the training set. The state-of-the-art method leverages conversations initiated by posts to enrich contextual information for the short posts. However, it is unrealistic to assume the existence of conversations before the hashtag annotation itself. Therefore, we propose to leverage news articles published before the microblog post to generate hashtags following a Retriever-Generator framework. Extensive experiments on English Twitter datasets demonstrate superior performance and significant advantages of leveraging news articles to generate hashtags.