On the Generalization of the C-Bound to Structured Output Ensemble Methods

TL;DR

This paper extends the PAC-Bayesian C-bound to structured output ensemble methods, enabling risk bounds for complex outputs like multiclass and multilabel, and facilitating new ensemble approaches with theoretical guarantees.

Contribution

It generalizes the C-bound to structured outputs, allowing risk assessment and development of ensemble methods with PAC-Bayesian guarantees for complex output spaces.

Findings

The generalized C-bound applies to multiclass and multilabel outputs.

It incorporates margin relaxations for flexible risk bounds.

Provides theoretical foundation for new structured output ensemble methods.

Abstract

This paper generalizes an important result from the PAC-Bayesian literature for binary classification to the case of ensemble methods for structured outputs. We prove a generic version of the \Cbound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second statistical moments of the vote's margin. This bound may advantageously $(i)$ be applied on more complex outputs such as multiclass labels and multilabel, and $(ii)$ allow to consider margin relaxations. These results open the way to develop new ensemble methods for structured output prediction with PAC-Bayesian guarantees.