# On the Convergence of Extended Variational Inference for Non-Gaussian Statistical Models

## TL;DR

This paper analyzes the convergence properties of extended variational inference (EVI) in non-Gaussian models, comparing approximation strategies and conditions, and demonstrating the advantages of the single lower-bound approach through theoretical and experimental results.

## Contribution

It clarifies the convergence conditions of EVI, compares approximation strategies, and highlights the benefits of the single lower-bound method in non-Gaussian Bayesian models.

## Key findings

- SLB approximation shows superior convergence properties.
- Weak and strong conditions have distinct theoretical implications.
- Extensive experiments validate the advantages of the SLB approach.

## Abstract

Variational inference (VI) is a widely used framework in Bayesian estimation. For most of the non-Gaussian statistical models, it is infeasible to find an analytically tractable solution to estimate the posterior distributions of the parameters. Recently, an improved framework, namely the extended variational inference (EVI), has been introduced and applied to derive analytically tractable solution by employing lower-bound approximation to the variational objective function. Two conditions required for EVI implementation, namely the weak condition and the strong condition, are discussed and compared in this paper. In practical implementation, the convergence of the EVI depends on the selection of the lower-bound approximation, no matter with the weak condition or the strong condition. In general, two approximation strategies, the single lower-bound (SLB) approximation and the multiple lower-bounds (MLB) approximation, can be applied to carry out the lower-bound approximation. To clarify the differences between the SLB and the MLB, we will also discuss the convergence properties of the aforementioned two approximations. Extensive comparisons are made based on some existing EVI-based non-Gaussian statistical models. Theoretical analysis are conducted to demonstrate the differences between the weak and the strong conditions. Qualitative and quantitative experimental results are presented to show the advantages of the SLB approximation.