# Gaussian Mean Field Regularizes by Limiting Learned Information

## TL;DR

This paper explains how Gaussian mean field regularization improves neural network generalization by limiting mutual information between parameters and data, supported by theoretical analysis and experiments.

## Contribution

It provides a theoretical understanding of how mean field inference regularizes models by controlling mutual information, linking it to generalization performance.

## Key findings

- Mean field inference limits mutual information, improving generalization.
- Bounding information acts as an effective regularizer in neural networks.
- Theoretical bounds relate posterior variance to generalization error.

## Abstract

Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show how mean field inference improves generalization by limiting mutual information between learned parameters and the data through noise. We quantify a maximum capacity when the posterior variance is either fixed or learned and connect it to generalization error, even when the KL-divergence in the objective is rescaled. Our experiments demonstrate that bounding information between parameters and data effectively regularizes neural networks on both supervised and unsupervised tasks.