Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection

TL;DR

This paper introduces PGOS, a reinforcement learning-based framework that synthesizes informative outliers in graph data to enhance out-of-distribution detection for Graph Neural Networks, achieving state-of-the-art results.

Contribution

The paper presents a novel RL-guided outlier synthesis method that adaptively explores low-density regions in graph latent space for improved OOD detection.

Findings

PGOS outperforms existing methods on multiple benchmarks.

It effectively explores informative OOD regions.

The approach enhances the robustness of graph OOD detectors.

Abstract

Detecting out-of-distribution (OOD) graphs is crucial for ensuring the safety and reliability of Graph Neural Networks. In unsupervised graph-level OOD detection, models are typically trained using only in-distribution (ID) data, resulting in incomplete feature space characterization and weak decision boundaries. Although synthesizing outliers offers a promising solution, existing approaches rely on fixed, non-adaptive sampling heuristics (e.g., distance- or density-based), limiting their ability to explore informative OOD regions. We propose a Policy-Guided Outlier Synthesis (PGOS) framework that replaces static heuristics with a learned exploration strategy. Specifically, PGOS trains a reinforcement learning agent to navigate low-density regions in a structured latent space and sample representations that most effectively refine the OOD decision boundary. These representations are then decoded into high-quality pseudo-OOD graphs to improve detector robustness. Extensive experiments demonstrate that PGOS achieves state-of-the-art performance on multiple graph OOD and anomaly detection benchmarks.