Majority Vote of Diverse Classifiers for Late Fusion

TL;DR

This paper introduces a novel late fusion method for multimedia indexing that optimally combines diverse classifiers using a quadratic program based on PAC-Bayesian theory, improving ranking performance.

Contribution

It extends MinCq to multimedia indexing, incorporating a pairwise ranking loss to enhance mean average precision while maintaining classifier diversity.

Findings

Improved mean average precision on PASCAL VOC'07

Effective late fusion of diverse classifiers

The proposed method outperforms existing approaches

Abstract

In the past few years, a lot of attention has been devoted to multimedia indexing by fusing multimodal informations. Two kinds of fusion schemes are generally considered: The early fusion and the late fusion. We focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle this problem, we investigate a recent and elegant well-founded quadratic program named MinCq coming from the machine learning PAC-Bayesian theory. MinCq looks for the weighted combination, over a set of real-valued functions seen as voters, leading to the lowest misclassification rate, while maximizing the voters' diversity. We propose an extension of MinCq tailored to multimedia indexing. Our method is based on an order-preserving pairwise loss adapted to ranking that allows us to improve Mean Averaged Precision measure while taking into account the diversity of the voters that we want to fuse. We provide evidence that this method is naturally adapted to late fusion procedures and confirm the good behavior of our approach on the challenging PASCAL VOC'07 benchmark.