Unsupervised MKL in Multi-layer Kernel Machines

TL;DR

This paper introduces an unsupervised multiple kernel learning approach within multi-layer kernel machines, enhancing data representation and classifier performance on noisy MNIST datasets.

Contribution

It proposes using multiple kernels in each layer of MKMs through an unsupervised convex combination strategy, extending prior single-kernel methods.

Findings

Improved data representation with MKL in MKMs.

Enhanced classifier accuracy on noisy MNIST datasets.

Demonstrated effectiveness of unsupervised kernel combination.

Abstract

Kernel based Deep Learning using multi-layer kernel machines(MKMs) was proposed by Y.Cho and L.K. Saul in \cite{saul}. In MKMs they used only one kernel(arc-cosine kernel) at a layer for the kernel PCA-based feature extraction. We propose to use multiple kernels in each layer by taking a convex combination of many kernels following an unsupervised learning strategy. Empirical study is conducted on \textit{mnist-back-rand}, \textit{mnist-back-image} and \textit{mnist-rot-back-image} datasets generated by adding random noise in the image background of MNIST dataset. Experimental results indicate that using MKL in MKMs earns a better representation of the raw data and improves the classifier performance.