Uncertainty in Neural Processes

TL;DR

This paper investigates how different neural network architectures and training objectives affect the quality of posterior inference in neural processes, especially when conditioning data is scarce, and introduces improvements for low-data scenarios.

Contribution

The work identifies specific architectural and objective modifications that enhance posterior inference in neural processes under low-data conditions.

Findings

Improved posterior predictive samples in image in-painting tasks

Certain pooling operators and variational families lead to better inference quality

Novel neural process architectures outperform existing models in low-data regimes

Abstract

We explore the effects of architecture and training objective choice on amortized posterior predictive inference in probabilistic conditional generative models. We aim this work to be a counterpoint to a recent trend in the literature that stresses achieving good samples when the amount of conditioning data is large. We instead focus our attention on the case where the amount of conditioning data is small. We highlight specific architecture and objective choices that we find lead to qualitative and quantitative improvement to posterior inference in this low data regime. Specifically we explore the effects of choices of pooling operator and variational family on posterior quality in neural processes. Superior posterior predictive samples drawn from our novel neural process architectures are demonstrated via image completion/in-painting experiments.