Out of Distribution Detection, Generalization, and Robustness Triangle with Maximum Probability Theorem

TL;DR

This paper introduces the Maximum Probability Theorem (MPT) as a regularization technique to improve out-of-distribution detection, generalization, and robustness in CNNs and energy-based models for computer vision tasks.

Contribution

It applies MPT as a novel regularization method to enhance model stability, generalization, and OOD detection performance across multiple datasets.

Findings

MPT-based regularization improves OOD detection accuracy.

The method stabilizes training and enhances model robustness.

Effective across various hyperparameters and model types.

Abstract

Maximum Probability Framework, powered by Maximum Probability Theorem, is a recent theoretical development in artificial intelligence, aiming to formally define probabilistic models, guiding development of objective functions, and regularization of probabilistic models. MPT uses the probability distribution that the models assume on random variables to provide an upper bound on the probability of the model. We apply MPT to challenging out-of-distribution (OOD) detection problems in computer vision by incorporating MPT as a regularization scheme in the training of CNNs and their energy-based variants. We demonstrate the effectiveness of the proposed method on 1080 trained models, with varying hyperparameters, and conclude that the MPT-based regularization strategy stabilizes and improves the generalization and robustness of base models in addition to enhanced OOD performance on CIFAR10, CIFAR100, and MNIST datasets.