Characterizing the Influence of Graph Elements

TL;DR

This paper develops influence functions for simple graph convolution models to understand the impact of removing nodes or edges, providing insights into model interpretability, robustness, and adversarial attack strategies on GCNs.

Contribution

It formulates influence functions for GCNs, analyzes their error bounds, and demonstrates their use in interpretability and adversarial attack scenarios.

Findings

Influence functions accurately estimate the impact of node/edge removal.

The method guides effective adversarial attacks on GCNs.

Influence estimation correlates well with actual model performance changes.

Abstract

Influence function, a method from robust statistics, measures the changes of model parameters or some functions about model parameters concerning the removal or modification of training instances. It is an efficient and useful post-hoc method for studying the interpretability of machine learning models without the need for expensive model re-training. Recently, graph convolution networks (GCNs), which operate on graph data, have attracted a great deal of attention. However, there is no preceding research on the influence functions of GCNs to shed light on the effects of removing training nodes/edges from an input graph. Since the nodes/edges in a graph are interdependent in GCNs, it is challenging to derive influence functions for GCNs. To fill this gap, we started with the simple graph convolution (SGC) model that operates on an attributed graph and formulated an influence function to approximate the changes in model parameters when a node or an edge is removed from an attributed graph. Moreover, we theoretically analyzed the error bound of the estimated influence of removing an edge. We experimentally validated the accuracy and effectiveness of our influence estimation function. In addition, we showed that the influence function of an SGC model could be used to estimate the impact of removing training nodes/edges on the test performance of the SGC without re-training the model. Finally, we demonstrated how to use influence functions to guide the adversarial attacks on GCNs effectively.