PAC-Bayes Analysis of Multi-view Learning

TL;DR

This paper develops eight PAC-Bayes bounds to analyze the generalization of multi-view classifiers, incorporating data-dependent priors and semi-supervised data, and compares their effectiveness on benchmark datasets.

Contribution

It introduces new PAC-Bayes bounds tailored for multi-view learning, including semi-supervised scenarios, with novel techniques like logarithmic determinant inequalities.

Findings

Multi-view bounds outperform single-view bounds on benchmarks.

Semi-supervised bounds effectively utilize unlabeled data.

New inequalities improve the tightness of generalization bounds.

Abstract

This paper presents eight PAC-Bayes bounds to analyze the generalization performance of multi-view classifiers. These bounds adopt data dependent Gaussian priors which emphasize classifiers with high view agreements. The center of the prior for the first two bounds is the origin, while the center of the prior for the third and fourth bounds is given by a data dependent vector. An important technique to obtain these bounds is two derived logarithmic determinant inequalities whose difference lies in whether the dimensionality of data is involved. The centers of the fifth and sixth bounds are calculated on a separate subset of the training set. The last two bounds use unlabeled data to represent view agreements and are thus applicable to semi-supervised multi-view learning. We evaluate all the presented multi-view PAC-Bayes bounds on benchmark data and compare them with previous single-view PAC-Bayes bounds. The usefulness and performance of the multi-view bounds are discussed.