# The Epistemic Uncertainty Gradient in Spaces of Random Projections

## TL;DR

This paper introduces a new way to use uncertainty estimates in machine learning to better understand and work with data distributions.

## Contribution

The novel contribution is reinterpreting epistemic uncertainty as a model of the data distribution using random projections.

## Key findings

- The approach enables efficient and parameter-free representation of arbitrary data distributions.
- Minimizing uncertainty via gradient descent helps find data points close to an initial input within the distribution.
- Applications include local Gaussian approximations, regression, and data unlearning.

## Abstract

This work presents a novel approach to handling epistemic uncertainty estimates with motivation from Bayesian linear regression. We propose treating the model-dependent variance in the predictive distribution—commonly associated with epistemic uncertainty—as a model for the underlying data distribution. Using high-dimensional random feature transformations, this approach allows for a computationally efficient, parameter-free representation of arbitrary data distributions. This allows assessing whether a query point lies within the distribution, which can also provide insights into outlier detection and generalization tasks. Furthermore, given an initial input, minimizing the uncertainty using gradient descent offers a new method of querying data points that are close to the initial input and belong to the distribution resembling the training data, much like auto-completion in associative networks. We extend the proposed method to applications such as local Gaussian approximations, input–output regression, and even a mechanism for unlearning of data. This reinterpretation of uncertainty, alongside the geometric insights it provides, offers an innovative and novel framework for addressing classical machine learning challenges.