Evaluation and Comparison of Diffusion Models with Motif Features

TL;DR

This paper evaluates diffusion models by analyzing the motif features of generated temporal networks, revealing that the diffusion process significantly influences motif characteristics more than network topology.

Contribution

It introduces a motif-based evaluation framework with benchmarks and metrics to compare diffusion models, highlighting the dominance of diffusion process over network topology.

Findings

Motif features are primarily determined by the diffusion model.

Proposed benchmarks measure stability and separability of diffusion models.

Diffusion models often ignore underlying network topology.

Abstract

Diffusion models simulate the propagation of influence in networks. The design and evaluation of diffusion models has been subjective and empirical. When being applied to a network represented by a graph, the diffusion model generates a sequence of edges on which the influence flows, such sequence forms a temporal network. In most scenarios, the statistical properties or the characteristics of a network are inferred by analyzing the temporal networks generated by diffusion models. To analyze real temporal networks, the motif has been proposed as a reliable feature. However, it is unclear how the network topology and the diffusion model affect the motif feature of a generated temporal network. In this paper, we adopt the motif feature to evaluate the temporal graph generated by a diffusion model, thence the diffusion model itself. Two benchmarks for quantitively evaluating diffusion models with motif, stability and separability, are proposed and measured on numerous diffusion models. One motif-based metric is proposed to measure the similarity between diffusion models. The experiments suggest that the motif of a generated temporal network is dominated by the diffusion model, while the network topology is almost ignored. This result indicates that more practical and reliable diffusion models have to be designed with delicacy in order to capture the propagation patterns of real temporal networks.