# MaxEntropy Pursuit Variational Inference

## TL;DR

This paper introduces MaxEntropy Pursuit, a variational inference method that uses a greedy approach with tractable base learners and a Max-Entropy framework to better capture complex, multimodal posterior distributions in neural networks.

## Contribution

It presents a novel greedy variational inference technique leveraging Max-Entropy to improve approximation of complex posteriors in neural network models.

## Key findings

- Effective in capturing multimodal posteriors
- Demonstrates improved inference in continual learning settings
- Utilizes tractable base learners for scalable inference

## Abstract

One of the core problems in variational inference is a choice of approximate posterior distribution. It is crucial to trade-off between efficient inference with simple families as mean-field models and accuracy of inference. We propose a variant of a greedy approximation of the posterior distribution with tractable base learners. Using Max-Entropy approach, we obtain a well-defined optimization problem. We demonstrate the ability of the method to capture complex multimodal posterior via continual learning setting for neural networks.