Predictive Uncertainty through Quantization

TL;DR

This paper introduces SQUAD, a scalable and flexible method for predictive uncertainty estimation using discretized latent variables, improving over traditional variational approaches in high-risk domains.

Contribution

The paper proposes SQUAD, a novel approach that employs quantized latent variables for better uncertainty estimation in deep models, addressing overconfidence issues of prior methods.

Findings

SQUAD provides competitive predictive uncertainty estimates.

The method learns complex non-linearities effectively.

It is scalable, self-normalizing, and sample efficient.

Abstract

High-risk domains require reliable confidence estimates from predictive models. Deep latent variable models provide these, but suffer from the rigid variational distributions used for tractable inference, which err on the side of overconfidence. We propose Stochastic Quantized Activation Distributions (SQUAD), which imposes a flexible yet tractable distribution over discretized latent variables. The proposed method is scalable, self-normalizing and sample efficient. We demonstrate that the model fully utilizes the flexible distribution, learns interesting non-linearities, and provides predictive uncertainty of competitive quality.