Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance
Nikos Kanakaris, Heng Ping, Xiongye Xiao, Nesreen K. Ahmed, Luca, Luceri, Emilio Ferrara, Paul Bogdan

TL;DR
This paper introduces a novel LLM-based framework using prompt engineering and Balanced RAG to detect organized disinformation campaigns on social media, effectively addressing class imbalance without model fine-tuning.
Contribution
The paper presents a new approach leveraging prompt engineering and Balanced RAG with LLMs for campaign detection, avoiding the need for training or fine-tuning.
Findings
Outperforms traditional graph-based methods in precision, recall, and F1 scores.
Achieves 2x-3x improvements over baselines.
Effectively handles class imbalance in campaign detection.
Abstract
Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the textual similarities of their posts. Despite their effectiveness in recognizing astroturf campaigns, these methods face significant challenges, notably the class imbalance in available training datasets. To mitigate this issue, recent methods usually resort to data augmentation or increasing the number of positive samples, which may not always be feasible or sufficient in real-world settings. Following a different path, in this paper, we propose a novel framework for identifying astroturf…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image Processing Techniques
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Layer Normalization · Dense Connections · Adam · Softmax · Linear Warmup With Linear Decay · Residual Connection · Dropout · Byte Pair Encoding
