Learning optimal Bayesian prior probabilities from data

TL;DR

This paper introduces a machine learning approach to learn optimal Bayesian prior probabilities from data, outperforming traditional uniform priors in text classification tasks.

Contribution

It proposes a novel method for learning priors from data by maximizing a target function, demonstrated through Wikipedia article categorization experiments.

Findings

Study models outperformed baseline models with statistical significance.

Performance improvement up to 443%, average 193%.

Learned priors significantly improved classification accuracy.

Abstract

Noninformative uniform priors are staples of Bayesian inference, especially in Bayesian machine learning. This study challenges the assumption that they are optimal and their use in Bayesian inference yields optimal outcomes. Instead of using arbitrary noninformative uniform priors, we propose a machine learning based alternative method, learning optimal priors from data by maximizing a target function of interest. Applying na\"ive Bayes text classification methodology and a search algorithm developed for this study, our system learned priors from data using the positive predictive value metric as the target function. The task was to find Wikipedia articles that had not (but should have) been categorized under certain Wikipedia categories. We conducted five sets of experiments using separate Wikipedia categories. While the baseline models used the popular Bayes-Laplace priors, the study models learned the optimal priors for each set of experiments separately before using them. The results showed that the study models consistently outperformed the baseline models with a wide margin of statistical significance (p < 0.001). The measured performance improvement of the study model over the baseline was as high as 443% with the mean value of 193% over five Wikipedia categories.