Non-Volatile Memory Accelerated Posterior Estimation

TL;DR

This paper introduces a method leveraging non-volatile memory to efficiently approximate complex posterior distributions, enabling better uncertainty estimation and improved downstream task predictions in machine learning.

Contribution

It demonstrates how high-capacity persistent storage can make feasible the approximation of large, complex posterior distributions, enhancing Bayesian inference.

Findings

Improved predictive accuracy in downstream tasks.

Feasibility of approximating large posterior distributions.

Enhanced uncertainty estimation in models.

Abstract

Bayesian inference allows machine learning models to express uncertainty. Current machine learning models use only a single learnable parameter combination when making predictions, and as a result are highly overconfident when their predictions are wrong. To use more learnable parameter combinations efficiently, these samples must be drawn from the posterior distribution. Unfortunately computing the posterior directly is infeasible, so often researchers approximate it with a well known distribution such as a Gaussian. In this paper, we show that through the use of high-capacity persistent storage, models whose posterior distribution was too big to approximate are now feasible, leading to improved predictions in downstream tasks.