LMKL-Net: A Fast Localized Multiple Kernel Learning Solver via Deep Neural Networks

TL;DR

This paper introduces LMKL-Net, a deep neural network approach for localized multiple kernel learning that improves accuracy and training speed over existing methods by using attentional networks and multilayer perceptrons.

Contribution

The paper presents a novel neural network-based solver for LMKL that parameterizes kernel gating and classification, enabling faster training and better accuracy.

Findings

Outperforms state-of-the-art MKL solvers in accuracy.

Training is approximately 100 times faster.

Requires significantly less memory for large-scale datasets.

Abstract

In this paper we propose solving localized multiple kernel learning (LMKL) using LMKL-Net, a feedforward deep neural network. In contrast to previous works, as a learning principle we propose {\em parameterizing} both the gating function for learning kernel combination weights and the multiclass classifier in LMKL using an attentional network (AN) and a multilayer perceptron (MLP), respectively. In this way we can learn the (nonlinear) decision function in LMKL (approximately) by sequential applications of AN and MLP. Empirically on benchmark datasets we demonstrate that overall LMKL-Net can not only outperform the state-of-the-art MKL solvers in terms of accuracy, but also be trained about {\em two orders of magnitude} faster with much smaller memory footprint for large-scale learning.