Can Dense Connectivity Benefit Outlier Detection? An Odyssey with NAS

TL;DR

This paper introduces DCSOD, a neural architecture search method that optimizes dense connectivity in CNNs for improved out-of-distribution detection, achieving state-of-the-art results on CIFAR benchmarks.

Contribution

It proposes a novel NAS framework that explores dense connectivity patterns in CNNs specifically for OOD detection, with a new evaluation stabilization technique.

Findings

DCSOD outperforms existing architectures and NAS baselines.

Achieves approximately 1% AUROC improvement on CIFAR benchmarks.

Demonstrates the effectiveness of dense connectivity in OOD detection.

Abstract

Recent advances in Out-of-Distribution (OOD) Detection is the driving force behind safe and reliable deployment of Convolutional Neural Networks (CNNs) in real world applications. However, existing studies focus on OOD detection through confidence score and deep generative model-based methods, without considering the impact of DNN structures, especially dense connectivity in architecture fabrications. In addition, existing outlier detection approaches exhibit high variance in generalization performance, lacking stability and confidence in evaluating and ranking different outlier detectors. In this work, we propose a novel paradigm, Dense Connectivity Search of Outlier Detector (DCSOD), that automatically explore the dense connectivity of CNN architectures on near-OOD detection task using Neural Architecture Search (NAS). We introduce a hierarchical search space containing versatile convolution operators and dense connectivity, allowing a flexible exploration of CNN architectures with diverse connectivity patterns. To improve the quality of evaluation on OOD detection during search, we propose evolving distillation based on our multi-view feature learning explanation. Evolving distillation stabilizes training for OOD detection evaluation, thus improves the quality of search. We thoroughly examine DCSOD on CIFAR benchmarks under OOD detection protocol. Experimental results show that DCSOD achieve remarkable performance over widely used architectures and previous NAS baselines. Notably, DCSOD achieves state-of-the-art (SOTA) performance on CIFAR benchmark, with AUROC improvement of $\sim$1.0%.