Towards Realistic Synthetic User-Generated Content: A Scaffolding Approach to Generating Online Discussions

TL;DR

This paper presents a novel scaffolding approach to generate realistic large-scale synthetic online discussion data using LLMs, addressing limitations of straightforward generation and enhancing data realism for machine learning applications.

Contribution

It introduces a multi-step generation framework with discussion scaffolds, improving the realism and control of synthetic social media content creation.

Findings

Framework successfully generates realistic discussion threads

Evaluation measures effectively compare synthetic data quality

Adaptable to different social media platforms

Abstract

The emergence of synthetic data represents a pivotal shift in modern machine learning, offering a solution to satisfy the need for large volumes of data in domains where real data is scarce, highly private, or difficult to obtain. We investigate the feasibility of creating realistic, large-scale synthetic datasets of user-generated content, noting that such content is increasingly prevalent and a source of frequently sought information. Large language models (LLMs) offer a starting point for generating synthetic social media discussion threads, due to their ability to produce diverse responses that typify online interactions. However, as we demonstrate, straightforward application of LLMs yields limited success in capturing the complex structure of online discussions, and standard prompting mechanisms lack sufficient control. We therefore propose a multi-step generation process, predicated on the idea of creating compact representations of discussion threads, referred to as scaffolds. Our framework is generic yet adaptable to the unique characteristics of specific social media platforms. We demonstrate its feasibility using data from two distinct online discussion platforms. To address the fundamental challenge of ensuring the representativeness and realism of synthetic data, we propose a portfolio of evaluation measures to compare various instantiations of our framework.