Efficient and Adaptive Kernelization for Nonlinear Max-margin Multi-view Learning

TL;DR

This paper introduces an adaptive, scalable kernelized multi-view learning model that leverages Bayesian methods and random Fourier features to improve classification performance without heavy kernel tuning or Gram matrix computations.

Contribution

It proposes a novel Bayesian framework that adaptively learns shift-invariant kernels from data using random Fourier features and Dirichlet process Gaussian mixtures, with an efficient inference algorithm.

Findings

Outperforms existing multi-view learning methods on real datasets.

Scales linearly with training data size, reducing computational resources.

Effectively learns kernels adaptively without manual tuning.

Abstract

Existing multi-view learning methods based on kernel function either require the user to select and tune a single predefined kernel or have to compute and store many Gram matrices to perform multiple kernel learning. Apart from the huge consumption of manpower, computation and memory resources, most of these models seek point estimation of their parameters, and are prone to overfitting to small training data. This paper presents an adaptive kernel nonlinear max-margin multi-view learning model under the Bayesian framework. Specifically, we regularize the posterior of an efficient multi-view latent variable model by explicitly mapping the latent representations extracted from multiple data views to a random Fourier feature space where max-margin classification constraints are imposed. Assuming these random features are drawn from Dirichlet process Gaussian mixtures, we can adaptively learn shift-invariant kernels from data according to Bochners theorem. For inference, we employ the data augmentation idea for hinge loss, and design an efficient gradient-based MCMC sampler in the augmented space. Having no need to compute the Gram matrix, our algorithm scales linearly with the size of training set. Extensive experiments on real-world datasets demonstrate that our method has superior performance.