TSSuBERT: Tweet Stream Summarization Using BERT
Alexis Dusart, Karen Pinel-Sauvagnat, Gilles Hubert

TL;DR
This paper introduces a new large dataset for Twitter event summarization and proposes a neural extractive model that combines BERT and vocabulary frequency to effectively summarize large tweet streams, adapting output size dynamically.
Contribution
The paper provides the first large dataset for Twitter summarization and a novel neural model that integrates pre-trained language models with frequency-based features for adaptive tweet stream summarization.
Findings
Promising results compared to state-of-the-art baselines.
Model adapts summary size automatically based on input.
Effective summarization of large tweet streams.
Abstract
The development of deep neural networks and the emergence of pre-trained language models such as BERT allow to increase performance on many NLP tasks. However, these models do not meet the same popularity for tweet summarization, which can probably be explained by the lack of existing collections for training and evaluation. Our contribution in this paper is twofold : (1) we introduce a large dataset for Twitter event summarization, and (2) we propose a neural model to automatically summarize huge tweet streams. This extractive model combines in an original way pre-trained language models and vocabulary frequency-based representations to predict tweet salience. An additional advantage of the model is that it automatically adapts the size of the output summary according to the input tweet stream. We conducted experiments using two different Twitter collections, and promising results are…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Advanced Text Analysis Techniques · Natural Language Processing Techniques
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Adam · Layer Normalization · Refunds@Expedia|||How do I get a full refund from Expedia? · Linear Warmup With Linear Decay · Residual Connection · WordPiece · Attention Dropout
