Synthesizing Diverse Network Flow Datasets with Scalable Dynamic Multigraph Generation

TL;DR

This paper presents a scalable machine learning approach for generating high-fidelity synthetic network flow datasets using dynamic multigraphs, improving accuracy and diversity over previous methods.

Contribution

Introduces a novel scalable model combining stochastic Kronecker graphs, GANs, and XGBoost for realistic synthetic network datasets with new evaluation metrics.

Findings

Outperforms previous graph generation methods in accuracy

Maintains high diversity in synthetic datasets

Provides new metrics for evaluating graph generative models

Abstract

Obtaining real-world network datasets is often challenging because of privacy, security, and computational constraints. In the absence of such datasets, graph generative models become essential tools for creating synthetic datasets. In this paper, we introduce a novel machine learning model for generating high-fidelity synthetic network flow datasets that are representative of real-world networks. Our approach involves the generation of dynamic multigraphs using a stochastic Kronecker graph generator for structure generation and a tabular generative adversarial network for feature generation. We further employ an XGBoost (eXtreme Gradient Boosting) model for graph alignment, ensuring accurate overlay of features onto the generated graph structure. We evaluate our model using new metrics that assess both the accuracy and diversity of the synthetic graphs. Our results demonstrate improvements in accuracy over previous large-scale graph generation methods while maintaining similar efficiency. We also explore the trade-off between accuracy and diversity in synthetic graph dataset creation, a topic not extensively covered in related works. Our contributions include the synthesis and evaluation of large real-world netflow datasets and the definition of new metrics for evaluating synthetic graph generative models.