Anomalies by Synthesis: Anomaly Detection using Generative Diffusion Models for Off-Road Navigation

TL;DR

This paper introduces a novel test-time anomaly detection method for off-road navigation using generative diffusion models to synthesize images and identify anomalies through pixel-wise analysis, without requiring retraining.

Contribution

It presents a new analysis-by-synthesis approach leveraging diffusion models for pixel-wise anomaly detection in unstructured environments, avoiding retraining or fine-tuning.

Findings

Effective anomaly detection in off-road environments

No retraining or fine-tuning needed for the diffusion model

Combines vision-language models for semantic anomaly detection

Abstract

In order to navigate safely and reliably in off-road and unstructured environments, robots must detect anomalies that are out-of-distribution (OOD) with respect to the training data. We present an analysis-by-synthesis approach for pixel-wise anomaly detection without making any assumptions about the nature of OOD data. Given an input image, we use a generative diffusion model to synthesize an edited image that removes anomalies while keeping the remaining image unchanged. Then, we formulate anomaly detection as analyzing which image segments were modified by the diffusion model. We propose a novel inference approach for guided diffusion by analyzing the ideal guidance gradient and deriving a principled approximation that bootstraps the diffusion model to predict guidance gradients. Our editing technique is purely test-time that can be integrated into existing workflows without the need for retraining or fine-tuning. Finally, we use a combination of vision-language foundation models to compare pixels in a learned feature space and detect semantically meaningful edits, enabling accurate anomaly detection for off-road navigation. Project website: https://siddancha.github.io/anomalies-by-diffusion-synthesis/