NP-PROV: Neural Processes with Position-Relevant-Only Variances

TL;DR

NP-PROV introduces a novel neural process model that separates position-related uncertainty from function value uncertainty, improving out-of-domain performance and achieving state-of-the-art likelihood on various datasets.

Contribution

It proposes a new neural process model that derives mean and variance from separate latent spaces, enhancing out-of-domain uncertainty modeling.

Findings

Achieves state-of-the-art likelihood on synthetic and real-world datasets.

Maintains bounded variance under function drift conditions.

Outperforms existing neural process models in out-of-domain tasks.

Abstract

Neural Processes (NPs) families encode distributions over functions to a latent representation, given context data, and decode posterior mean and variance at unknown locations. Since mean and variance are derived from the same latent space, they may fail on out-of-domain tasks where fluctuations in function values amplify the model uncertainty. We present a new member named Neural Processes with Position-Relevant-Only Variances (NP-PROV). NP-PROV hypothesizes that a target point close to a context point has small uncertainty, regardless of the function value at that position. The resulting approach derives mean and variance from a function-value-related space and a position-related-only latent space separately. Our evaluation on synthetic and real-world datasets reveals that NP-PROV can achieve state-of-the-art likelihood while retaining a bounded variance when drifts exist in the function value.