R\'enyi Neural Processes
Xuesong Wang, He Zhao, Edwin V. Bonilla

TL;DR
Rényi Neural Processes introduce a novel divergence measure to improve uncertainty modeling in neural processes, addressing prior misspecification and achieving superior performance in regression and image inpainting tasks.
Contribution
The paper proposes Rényi Neural Processes, replacing KL divergence with Rényi divergence to mitigate prior misspecification in neural processes, leading to enhanced performance.
Findings
RNPs outperform standard NPs in log-likelihoods.
RNPs show significant improvements in regression tasks.
RNPs excel in image inpainting benchmarks.
Abstract
Neural Processes (NPs) are deep probabilistic models that represent stochastic processes by conditioning their prior distributions on a set of context points. Despite their advantages in uncertainty estimation for complex distributions, NPs enforce parameterization coupling between the conditional prior model and the posterior model. We show that this coupling amounts to prior misspecification and revisit the NP objective to address this issue. More specifically, we propose R\'enyi Neural Processes (RNP), a method that replaces the standard KL divergence with the R\'enyi divergence, dampening the effects of the misspecified prior during posterior updates. We validate our approach across multiple benchmarks including regression and image inpainting tasks, and show significant performance improvements of RNPs in real-world problems. Our extensive experiments show consistently better…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsNeural Networks and Applications
MethodsSparse Evolutionary Training · Variational Inference · Inpainting
