Hybrid Open-set Segmentation with Synthetic Negative Data

TL;DR

This paper introduces a hybrid open-set segmentation method that combines generative and discriminative anomaly detection cues, enhancing performance with minimal computational cost.

Contribution

It proposes a novel anomaly scoring method that fuses generative and discriminative cues, applicable to any closed-set segmentation model, using negative data from various sources.

Findings

Strong open-set segmentation performance demonstrated

Effective fusion of generative and discriminative cues

Negligible additional computational overhead

Abstract

Open-set segmentation can be conceived by complementing closed-set classification with anomaly detection. Many of the existing dense anomaly detectors operate through generative modelling of regular data or by discriminating with respect to negative data. These two approaches optimize different objectives and therefore exhibit different failure modes. Consequently, we propose a novel anomaly score that fuses generative and discriminative cues. Our score can be implemented by upgrading any closed-set segmentation model with dense estimates of dataset posterior and unnormalized data likelihood. The resulting dense hybrid open-set models require negative training images that can be sampled from an auxiliary negative dataset, from a jointly trained generative model, or from a mixture of both sources. We evaluate our contributions on benchmarks for dense anomaly detection and open-set segmentation. The experiments reveal strong open-set performance in spite of negligible computational overhead.