Reject Illegal Inputs with Generative Classifier Derived from Any Discriminative Classifier

TL;DR

This paper introduces SDIM-logit, a method that enhances generative classifiers derived from discriminative classifiers to effectively reject illegal inputs like adversarial and out-of-distribution samples, improving detection performance.

Contribution

SDIM-logit modifies SDIM by leveraging logits from discriminative classifiers, inheriting their performance with minimal additional parameters and enabling efficient rejection of illegal inputs.

Findings

Significantly improves illegal input detection when rejecting uncertain samples.

Efficiently inherits discriminative classifier performance without retraining from scratch.

Effective against adversarial, corrupted, and out-of-distribution samples.

Abstract

Generative classifiers have been shown promising to detect illegal inputs including adversarial examples and out-of-distribution samples. Supervised Deep Infomax~(SDIM) is a scalable end-to-end framework to learn generative classifiers. In this paper, we propose a modification of SDIM termed SDIM-\emph{logit}. Instead of training generative classifier from scratch, SDIM-\emph{logit} first takes as input the logits produced any given discriminative classifier, and generate logit representations; then a generative classifier is derived by imposing statistical constraints on logit representations. SDIM-\emph{logit} could inherit the performance of the discriminative classifier without loss. SDIM-\emph{logit} incurs a negligible number of additional parameters, and can be efficiently trained with base classifiers fixed. We perform \emph{classification with rejection}, where test samples whose class conditionals are smaller than pre-chosen thresholds will be rejected without predictions. Experiments on illegal inputs, including adversarial examples, samples with common corruptions, and out-of-distribution~(OOD) samples show that allowed to reject a portion of test samples, SDIM-\emph{logit} significantly improves the performance on the left test sets.