Learning Hierarchical Feature Space Using CLAss-specific Subspace Multiple Kernel -- Metric Learning for Classification

TL;DR

This paper introduces a novel hierarchical kernel-based metric learning method called CLAS(M)K-ML that improves class separation, reduces computational complexity, and can be integrated as a preprocessing step for kernel methods.

Contribution

It proposes a pairwise-computation-free, flexible, and efficient kernel metric learning approach with a hierarchical extension for enhanced classification performance.

Findings

No pairwise computations required, reducing complexity.

Hierarchical structure improves classification accuracy.

Compatible as a preprocessing step for kernel methods.

Abstract

Metric learning for classification has been intensively studied over the last decade. The idea is to learn a metric space induced from a normed vector space on which data from different classes are well separated. Different measures of the separation thus lead to various designs of the objective function in the metric learning model. One classical metric is the Mahalanobis distance, where a linear transformation matrix is designed and applied on the original dataset to obtain a new subspace equipped with the Euclidean norm. The kernelized version has also been developed, followed by Multiple-Kernel learning models. In this paper, we consider metric learning to be the identification of the best kernel function with respect to a high class separability in the corresponding metric space. The contribution is twofold: 1) No pairwise computations are required as in most metric learning techniques; 2) Better flexibility and lower computational complexity is achieved using the CLAss-Specific (Multiple) Kernel - Metric Learning (CLAS(M)K-ML). The proposed techniques can be considered as a preprocessing step to any kernel method or kernel approximation technique. An extension to a hierarchical learning structure is also proposed to further improve the classification performance, where on each layer, the CLASMK is computed based on a selected "marginal" subset and feature vectors are constructed by concatenating the features from all previous layers.