Conjunction Subspaces Test for Conformal and Selective Classification

TL;DR

This paper introduces a novel classifier that combines significance tests over random subspaces to produce consensus p-values, enabling conformal and selective classification with uncertainty quantification.

Contribution

It proposes a new conjunction subspaces test-based classifier that integrates multiple hypothesis tests for improved conformal and selective classification.

Findings

The classifier effectively quantifies uncertainty via consensus p-values.

Theoretical generalization error bounds are established.

Empirical results demonstrate improved classification performance.

Abstract

In this paper, we present a new classifier, which integrates significance testing results over different random subspaces to yield consensus p-values for quantifying the uncertainty of classification decision. The null hypothesis is that the test sample has no association with the target class on a randomly chosen subspace, and hence the classification problem can be formulated as a problem of testing for the conjunction of hypotheses. The proposed classifier can be easily deployed for the purpose of conformal prediction and selective classification with reject and refine options by simply thresholding the consensus p-values. The theoretical analysis on the generalization error bound of the proposed classifier is provided and empirical studies on real data sets are conducted as well to demonstrate its effectiveness.