Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours

TL;DR

This paper introduces a novel generative model, AMDN-HAGE, for detecting coordinated malicious accounts on social media by modeling account activities and hidden group behaviors, addressing limitations of previous methods.

Contribution

The paper presents a new probabilistic model combining TPP and GMM to identify coordinated accounts without strict assumptions, along with a bilevel optimization algorithm with convergence guarantees.

Findings

Effectively detected coordinated campaigns on Twitter.

Identified high influence among coordinated account pairs.

Discovered coordinated groups spreading misinformation on COVID-19.

Abstract

Disinformation campaigns on social media, involving coordinated activities from malicious accounts towards manipulating public opinion, have become increasingly prevalent. Existing approaches to detect coordinated accounts either make very strict assumptions about coordinated behaviours, or require part of the malicious accounts in the coordinated group to be revealed in order to detect the rest. To address these drawbacks, we propose a generative model, AMDN-HAGE (Attentive Mixture Density Network with Hidden Account Group Estimation) which jointly models account activities and hidden group behaviours based on Temporal Point Processes (TPP) and Gaussian Mixture Model (GMM), to capture inherent characteristics of coordination which is, accounts that coordinate must strongly influence each other's activities, and collectively appear anomalous from normal accounts. To address the challenges of optimizing the proposed model, we provide a bilevel optimization algorithm with theoretical guarantee on convergence. We verified the effectiveness of the proposed method and training algorithm on real-world social network data collected from Twitter related to coordinated campaigns from Russia's Internet Research Agency targeting the 2016 U.S. Presidential Elections, and to identify coordinated campaigns related to the COVID-19 pandemic. Leveraging the learned model, we find that the average influence between coordinated account pairs is the highest.On COVID-19, we found coordinated group spreading anti-vaccination, anti-masks conspiracies that suggest the pandemic is a hoax and political scam.