Distance-informed Neural Processes

TL;DR

The Distance-informed Neural Process (DNP) enhances uncertainty estimation in neural models by integrating global and distance-aware local latent variables with bi-Lipschitz regularization, leading to better calibration and out-of-distribution detection.

Contribution

DNP introduces a novel combination of global and local latent variables with distance-preserving regularization to improve uncertainty estimation in neural processes.

Findings

DNP achieves superior uncertainty calibration in regression and classification.

DNP outperforms standard Neural Processes in predictive accuracy.

DNP effectively distinguishes in- from out-of-distribution data.

Abstract

We propose the Distance-informed Neural Process (DNP), a novel variant of Neural Processes that improves uncertainty estimation by combining global and distance-aware local latent structures. Standard Neural Processes (NPs) often rely on a global latent variable and struggle with uncertainty calibration and capturing local data dependencies. DNP addresses these limitations by introducing a global latent variable to model task-level variations and a local latent variable to capture input similarity within a distance-preserving latent space. This is achieved through bi-Lipschitz regularization, which bounds distortions in input relationships and encourages the preservation of relative distances in the latent space. This modeling approach allows DNP to produce better-calibrated uncertainty estimates and more effectively distinguish in- from out-of-distribution data. Empirical results demonstrate that DNP achieves strong predictive performance and improved uncertainty calibration across regression and classification tasks.