Online Multiple Kernel Learning for Structured Prediction

TL;DR

This paper introduces online proximal algorithms for multiple kernel learning in structured prediction, enabling scalable and efficient learning with proven theoretical guarantees and successful experiments in handwriting recognition and dependency parsing.

Contribution

It presents a novel online proximal algorithm for MKL that overcomes scalability issues of previous batch methods, with theoretical analysis and practical validation.

Findings

The proposed method achieves competitive accuracy in handwriting recognition.

It demonstrates effective dependency parsing results.

The algorithms have proven regret, convergence, and generalization bounds.

Abstract

Despite the recent progress towards efficient multiple kernel learning (MKL), the structured output case remains an open research front. Current approaches involve repeatedly solving a batch learning problem, which makes them inadequate for large scale scenarios. We propose a new family of online proximal algorithms for MKL (as well as for group-lasso and variants thereof), which overcomes that drawback. We show regret, convergence, and generalization bounds for the proposed method. Experiments on handwriting recognition and dependency parsing testify for the successfulness of the approach.