Interactive Event Sifting using Bayesian Graph Neural Networks

TL;DR

This paper presents a Bayesian Graph Neural Network approach for interactive social media post sifting, enhancing forensic event analysis by reducing manual annotation through active learning and pseudo-labeling.

Contribution

It introduces a novel BGNN-based method for multimodal event classification and evaluates active learning and pseudo-labeling to improve social media data sifting efficiency.

Findings

BGNNs are effective for social media event sifting.

Active learning and pseudo-labeling benefits depend on the setting.

Using unlabelled data from other events improves performance.

Abstract

Forensic analysts often use social media imagery and texts to understand important events. A primary challenge is the initial sifting of irrelevant posts. This work introduces an interactive process for training an event-centric, learning-based multimodal classification model that automates sanitization. We propose a method based on Bayesian Graph Neural Networks (BGNNs) and evaluate active learning and pseudo-labeling formulations to reduce the number of posts the analyst must manually annotate. Our results indicate that BGNNs are useful for social-media data sifting for forensics investigations of events of interest, the value of active learning and pseudo-labeling varies based on the setting, and incorporating unlabelled data from other events improves performance.