Spatial Autoregressive Modeling of DINOv3 Embeddings for Unsupervised Anomaly Detection

TL;DR

This paper introduces a spatial autoregressive model for DINOv3 embeddings that explicitly captures spatial dependencies, improving unsupervised anomaly detection efficiency and performance without large memory overhead.

Contribution

It proposes a novel 2D autoregressive CNN framework that models spatial relationships in patch embeddings, reducing memory use and inference time compared to existing prototype-based methods.

Findings

Achieves competitive anomaly detection accuracy on BMAD datasets.

Significantly reduces inference time and memory requirements.

Demonstrates effectiveness of spatial dependency modeling in UAD.

Abstract

DINO models provide rich patch-level representations that have recently enabled strong performance in unsupervised anomaly detection (UAD). Most existing methods extract patch embeddings from ``normal'' images and model them independently, ignoring spatial and neighborhood relationships between patches. This implicitly assumes that self-attention and positional encodings sufficiently encode contextual information within each patch embedding. In addition, the normative distribution is often modeled as memory banks or prototype-based representations, which require storing large numbers of features and performing costly comparisons at inference time, leading to substantial memory and computational overhead. In this work, we address both limitations by proposing a simple and efficient framework that explicitly models spatial and contextual dependencies between patch embeddings using a 2D autoregressive (AR) model. Instead of storing embeddings or clustering prototypes, our approach learns a compact parametric model of the normative distribution via an AR convolutional neural network (CNN). At test time, anomaly detection reduces to a single forward pass through the network and enables fast and memory-efficient inference. We evaluate our method on the BMAD benchmark, which comprises three medical imaging datasets, and compare it against existing work including recent DINO-based methods. Experimental results demonstrate that explicitly modeling spatial dependencies achieves competitive anomaly detection performance while substantially reducing inference time and memory requirements. Code is available at the project page: https://eerdil.github.io/spatial-ar-dinov3-uad/.