Perturb-and-Compare Approach for Detecting Out-of-Distribution Samples in Constrained Access Environments

TL;DR

This paper introduces MixDiff, a model-agnostic OOD detection method that compares output differences after input perturbations, effective even with restricted model access, improving safety in remote API-based ML systems.

Contribution

MixDiff is a novel, model-agnostic framework for OOD detection that works without access to model parameters or activations, enhancing safety in constrained environments.

Findings

Consistently improves OOD detection across vision and text datasets.

Effective even when model parameters are inaccessible.

Theoretically justified for detecting overconfident OOD samples.

Abstract

Accessing machine learning models through remote APIs has been gaining prevalence following the recent trend of scaling up model parameters for increased performance. Even though these models exhibit remarkable ability, detecting out-of-distribution (OOD) samples remains a crucial safety concern for end users as these samples may induce unreliable outputs from the model. In this work, we propose an OOD detection framework, MixDiff, that is applicable even when the model's parameters or its activations are not accessible to the end user. To bypass the access restriction, MixDiff applies an identical input-level perturbation to a given target sample and a similar in-distribution (ID) sample, then compares the relative difference in the model outputs of these two samples. MixDiff is model-agnostic and compatible with existing output-based OOD detection methods. We provide theoretical analysis to illustrate MixDiff's effectiveness in discerning OOD samples that induce overconfident outputs from the model and empirically demonstrate that MixDiff consistently enhances the OOD detection performance on various datasets in vision and text domains.