A Primer on PAC-Bayesian Learning

TL;DR

This paper provides a comprehensive survey of the PAC-Bayesian framework in machine learning, highlighting its theoretical foundations, algorithmic developments, and generalisation properties.

Contribution

It offers a self-contained overview of PAC-Bayesian learning, summarizing key theoretical insights and recent algorithmic advancements in the field.

Findings

PAC-Bayesian methods have strong generalisation guarantees.

Recent algorithms improve learning efficiency and performance.

The framework unifies Bayesian and frequentist approaches.

Abstract

Generalised Bayesian learning algorithms are increasingly popular in machine learning, due to their PAC generalisation properties and flexibility. The present paper aims at providing a self-contained survey on the resulting PAC-Bayes framework and some of its main theoretical and algorithmic developments.