nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods

TL;DR

This paper introduces nnOOD, a framework that standardizes the benchmarking of self-supervised anomaly localization methods in medical imaging, enabling fair comparison and evaluation across diverse datasets and tasks.

Contribution

The paper presents nnOOD, a novel framework that isolates the self-supervised task from training, facilitating fair comparison and efficient evaluation of anomaly localization methods.

Findings

State-of-the-art methods evaluated on X-ray dataset

Framework enables quick and faithful comparison of tasks

Isolating the task improves assessment of method performance

Abstract

The wide variety of in-distribution and out-of-distribution data in medical imaging makes universal anomaly detection a challenging task. Recently a number of self-supervised methods have been developed that train end-to-end models on healthy data augmented with synthetic anomalies. However, it is difficult to compare these methods as it is not clear whether gains in performance are from the task itself or the training pipeline around it. It is also difficult to assess whether a task generalises well for universal anomaly detection, as they are often only tested on a limited range of anomalies. To assist with this we have developed nnOOD, a framework that adapts nnU-Net to allow for comparison of self-supervised anomaly localisation methods. By isolating the synthetic, self-supervised task from the rest of the training process we perform a more faithful comparison of the tasks, whilst also making the workflow for evaluating over a given dataset quick and easy. Using this we have implemented the current state-of-the-art tasks and evaluated them on a challenging X-ray dataset.