Quantized Variational Inference
Amir Dib

TL;DR
Quantized Variational Inference introduces a novel algorithm that uses optimal Voronoi tessellation to achieve variance-free gradients for ELBO maximization, improving convergence speed with manageable bias.
Contribution
The paper proposes a new quantization-based variational inference method that reduces gradient variance and enhances convergence speed, with bias correction techniques.
Findings
Fast convergence for score function and reparametrized gradients
Variance-free gradients via Voronoi tessellation
Effective bias correction with Richardson extrapolation
Abstract
We present Quantized Variational Inference, a new algorithm for Evidence Lower Bound maximization. We show how Optimal Voronoi Tesselation produces variance free gradients for ELBO optimization at the cost of introducing asymptotically decaying bias. Subsequently, we propose a Richardson extrapolation type method to improve the asymptotic bound. We show that using the Quantized Variational Inference framework leads to fast convergence for both score function and the reparametrized gradient estimator at a comparable computational cost. Finally, we propose several experiments to assess the performance of our method and its limitations.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Stochastic Gradient Optimization Techniques · Statistical Methods and Inference
