Towards High-Fidelity Synthetic Multi-platform Social Media Datasets via Large Language Models

TL;DR

This paper investigates using large language models to generate high-quality synthetic social media datasets across multiple platforms, addressing data access constraints and enabling research in areas like disinformation and hate speech detection.

Contribution

It introduces multi-platform topic prompting and new fidelity metrics, demonstrating the potential of language models to produce realistic synthetic social media data.

Findings

Large language models can generate semantically relevant social media data

Different models vary in fidelity performance

Post-processing may enhance data quality

Abstract

Social media datasets are essential for research on a variety of topics, such as disinformation, influence operations, hate speech detection, or influencer marketing practices. However, access to social media datasets is often constrained due to costs and platform restrictions. Acquiring datasets that span multiple platforms, which is crucial for understanding the digital ecosystem, is particularly challenging. This paper explores the potential of large language models to create lexically and semantically relevant social media datasets across multiple platforms, aiming to match the quality of real data. We propose multi-platform topic-based prompting and employ various language models to generate synthetic data from two real datasets, each consisting of posts from three different social media platforms. We assess the lexical and semantic properties of the synthetic data and compare them with those of the real data. Our empirical findings show that using large language models to generate synthetic multi-platform social media data is promising, different language models perform differently in terms of fidelity, and a post-processing approach might be needed for generating high-fidelity synthetic datasets for research. In addition to the empirical evaluation of three state of the art large language models, our contributions include new fidelity metrics specific to multi-platform social media datasets.