Adapting Classifiers To Changing Class Priors During Deployment

TL;DR

This paper investigates methods for estimating and adapting to changing class priors during deployment, improving classifier accuracy in real-world scenarios with varying class distributions.

Contribution

It introduces techniques for estimating class priors from classifier outputs and demonstrates how incorporating these estimates enhances deployment accuracy.

Findings

Estimating class priors from classifier outputs is feasible.

Adapting classifiers to estimated priors improves accuracy.

Methods outperform static prior assumptions in dynamic environments.

Abstract

Conventional classifiers are trained and evaluated using balanced data sets in which all classes are equally present. Classifiers are now trained on large data sets such as ImageNet, and are now able to classify hundreds (if not thousands) of different classes. On one hand, it is desirable to train such general-purpose classifier on a very large number of classes so that it performs well regardless of the settings in which it is deployed. On the other hand, it is unlikely that all classes known to the classifier will occur in every deployment scenario, or that they will occur with the same prior probability. In reality, only a relatively small subset of the known classes may be present in a particular setting or environment. For example, a classifier will encounter mostly animals if its deployed in a zoo or for monitoring wildlife, aircraft and service vehicles at an airport, or various types of automobiles and commercial vehicles if it is used for monitoring traffic. Furthermore, the exact class priors are generally unknown and can vary over time. In this paper, we explore different methods for estimating the class priors based on the output of the classifier itself. We then show that incorporating the estimated class priors in the overall decision scheme enables the classifier to increase its run-time accuracy in the context of its deployment scenario.