When in Doubt: Improving Classification Performance with Alternating Normalization

TL;DR

This paper presents CAN, a post-processing normalization technique that enhances classification accuracy on difficult examples by leveraging high-confidence validation data, applicable to any probabilistic classifier.

Contribution

The paper introduces CAN, a novel non-parametric post-processing method that improves classification performance without significant computational costs.

Findings

CAN improves accuracy on challenging examples.

Effective across diverse classification tasks.

Minimal computational overhead.

Abstract

We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification. CAN improves classification accuracy for challenging examples by re-adjusting their predicted class probability distribution using the predicted class distributions of high-confidence validation examples. CAN is easily applicable to any probabilistic classifier, with minimal computation overhead. We analyze the properties of CAN using simulated experiments, and empirically demonstrate its effectiveness across a diverse set of classification tasks.