The Infinity Mirror Test for Analyzing the Robustness of Graph Generators

TL;DR

The paper introduces the infinity mirror test, a recursive method to evaluate the robustness and biases of graph generators by repeatedly fitting models to their own outputs, revealing degenerative patterns.

Contribution

It proposes a novel recursive stress test for graph generators, providing new insights into their biases and robustness beyond standard metrics.

Findings

Several common graph generators degenerate under the test

Degenerative patterns reveal biases in model assumptions

The test offers new directions for developing better graph models

Abstract

Graph generators learn a model from a source graph in order to generate a new graph that has many of the same properties. The learned models each have implicit and explicit biases built in, and its important to understand the assumptions that are made when generating a new graph. Of course, the differences between the new graph and the original graph, as compared by any number of graph properties, are important indicators of the biases inherent in any modelling task. But these critical differences are subtle and not immediately apparent using standard performance metrics. Therefore, we introduce the infinity mirror test for the analysis of graph generator performance and robustness. This stress test operates by repeatedly, recursively fitting a model to itself. A perfect graph generator would have no deviation from the original or ideal graph, however the implicit biases and assumptions that are cooked into the various models are exaggerated by the infinity mirror test allowing for new insights that were not available before. We show, via hundreds of experiments on 6 real world graphs, that several common graph generators do degenerate in interesting and informative ways. We believe that the observed degenerative patterns are clues to future development of better graph models.