BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs

TL;DR

This paper introduces BOND, a comprehensive benchmark for evaluating unsupervised outlier node detection methods on static attributed graphs, comparing 14 algorithms across multiple datasets and synthetic scenarios.

Contribution

It provides the first extensive benchmarking framework for unsupervised outlier detection on static attributed graphs, covering diverse methods, datasets, and synthetic data generation.

Findings

Graph neural networks perform competitively with classical methods.

Detection accuracy varies significantly across different datasets and outlier types.

The benchmark reveals trade-offs between detection performance, runtime, and memory usage.

Abstract

Detecting which nodes in graphs are outliers is a relatively new machine learning task with numerous applications. Despite the proliferation of algorithms developed in recent years for this task, there has been no standard comprehensive setting for performance evaluation. Consequently, it has been difficult to understand which methods work well and when under a broad range of settings. To bridge this gap, we present--to the best of our knowledge--the first comprehensive benchmark for unsupervised outlier node detection on static attributed graphs called BOND, with the following highlights. (1) We benchmark the outlier detection performance of 14 methods ranging from classical matrix factorization to the latest graph neural networks. (2) Using nine real datasets, our benchmark assesses how the different detection methods respond to two major types of synthetic outliers and separately to "organic" (real non-synthetic) outliers. (3) Using an existing random graph generation technique, we produce a family of synthetically generated datasets of different graph sizes that enable us to compare the running time and memory usage of the different outlier detection algorithms. Based on our experimental results, we discuss the pros and cons of existing graph outlier detection algorithms, and we highlight opportunities for future research. Importantly, our code is freely available and meant to be easily extendable: https://github.com/pygod-team/pygod/tree/main/benchmark