MTLTS: A Multi-Task Framework To Obtain Trustworthy Summaries From Crisis-Related Microblogs

TL;DR

This paper introduces MTLTS, an end-to-end multi-task framework that jointly assesses credibility and generates trustworthy summaries of crisis-related microblogs, improving accuracy and generalization across domains.

Contribution

It presents the first integrated solution combining credibility verification and summarization for crisis microblogs, utilizing hierarchical multi-task learning and structural conversation analysis.

Findings

Outperforms existing baselines in verification accuracy.

Achieves 21-35% higher verified ratio of summary tweets.

Attains 16-20% improvements in ROUGE1-F1 scores.

Abstract

Occurrences of catastrophes such as natural or man-made disasters trigger the spread of rumours over social media at a rapid pace. Presenting a trustworthy and summarized account of the unfolding event in near real-time to the consumers of such potentially unreliable information thus becomes an important task. In this work, we propose MTLTS, the first end-to-end solution for the task that jointly determines the credibility and summary-worthiness of tweets. Our credibility verifier is designed to recursively learn the structural properties of a Twitter conversation cascade, along with the stances of replies towards the source tweet. We then take a hierarchical multi-task learning approach, where the verifier is trained at a lower layer, and the summarizer is trained at a deeper layer where it utilizes the verifier predictions to determine the salience of a tweet. Different from existing disaster-specific summarizers, we model tweet summarization as a supervised task. Such an approach can automatically learn summary-worthy features, and can therefore generalize well across domains. When trained on the PHEME dataset [29], not only do we outperform the strongest baselines for the auxiliary task of verification/rumour detection, we also achieve 21 - 35% gains in the verified ratio of summary tweets, and 16 - 20% gains in ROUGE1-F1 scores over the existing state-of-the-art solutions for the primary task of trustworthy summarization.