DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detection

TL;DR

DEEDEE introduces a simple, efficient out-of-distribution detection method for reinforcement learning that outperforms complex models in accuracy and computational cost by leveraging minimal statistics.

Contribution

The paper presents DEEDEE, a minimalistic, fast OOD detector for RL that matches or exceeds existing methods with significantly reduced computational requirements.

Findings

DEEDEE achieves a 600-fold reduction in compute compared to baselines.

It delivers an average 5% absolute accuracy improvement.

Diverse anomalies imprint on RL trajectories through low-order statistics.

Abstract

Deploying reinforcement learning (RL) in safety-critical settings is constrained by brittleness under distribution shift. We study out-of-distribution (OOD) detection for RL time series and introduce DEEDEE, a two-statistic detector that revisits representation-heavy pipelines with a minimal alternative. DEEDEE uses only an episodewise mean and an RBF kernel similarity to a training summary, capturing complementary global and local deviations. Despite its simplicity, DEEDEE matches or surpasses contemporary detectors across standard RL OOD suites, delivering a 600-fold reduction in compute (FLOPs / wall-time) and an average 5% absolute accuracy gain over strong baselines. Conceptually, our results indicate that diverse anomaly types often imprint on RL trajectories through a small set of low-order statistics, suggesting a compact foundation for OOD detection in complex environments.