Attend and select: A segment selective transformer for microblog hashtag generation

TL;DR

This paper introduces a segment selective transformer with a novel segments selection mechanism to improve the quality and condensation of hashtags generated from microblog posts, outperforming existing methods.

Contribution

It proposes a new transformer model with a segments-selection procedure that effectively filters crucial tokens for better hashtag generation.

Findings

Significant improvements over baselines in evaluation metrics.

Effective modeling of different textual granularities.

Enhanced ability to generate condensed, relevant hashtags.

Abstract

Hashtag generation aims to generate short and informal topical tags from a microblog post, in which tokens or phrases form the hashtags. These tokens or phrases may originate from primary fragmental textual pieces (e.g., segments) in the original text and are separated into different segments. However, conventional sequence-to-sequence generation methods are hard to filter out secondary information from different textual granularity and are not good at selecting crucial tokens. Thus, they are suboptimal in generating more condensed hashtags. In this work, we propose a modified Transformer-based generation model with adding a segments-selection procedure for the original encoding and decoding phases. The segments-selection phase is based on a novel Segments Selection Mechanism (SSM) to model different textual granularity on global text, local segments, and tokens, contributing to generating condensed hashtags. Specifically, it first attends to primary semantic segments and then transforms discontinuous segments from the source text into a sequence of hashtags by selecting crucial tokens. Extensive evaluations on the two datasets reveal our approach's superiority with significant improvements to the extraction and generation baselines. The code and datasets are available at https://github.com/OpenSUM/HashtagGen.