PAC-Bayesian Majority Vote for Late Classifier Fusion

TL;DR

This paper introduces a PAC-Bayesian approach called MinCq for late classifier fusion in multimedia indexing, optimizing classifier weights to improve accuracy and ranking performance, validated on real image data.

Contribution

It extends MinCq with a pairwise loss for ranking, demonstrating its effectiveness for late fusion in multimedia indexing.

Findings

MinCq effectively optimizes classifier weights for late fusion.

Adding pairwise ranking loss improves Mean Averaged Precision.

Experiments confirm the method's good performance on real image benchmarks.

Abstract

A lot of attention has been devoted to multimedia indexing over the past few years. In the literature, we often consider two kinds of fusion schemes: The early fusion and the late fusion. In this paper we focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle this problem, we investigate a recent and elegant well-founded quadratic program named MinCq coming from the Machine Learning PAC-Bayes theory. MinCq looks for the weighted combination, over a set of real-valued functions seen as voters, leading to the lowest misclassification rate, while making use of the voters' diversity. We provide evidence that this method is naturally adapted to late fusion procedure. We propose an extension of MinCq by adding an order- preserving pairwise loss for ranking, helping to improve Mean Averaged Precision measure. We confirm the good behavior of the MinCq-based fusion approaches with experiments on a real image benchmark.