Kernelized Supervised Dictionary Learning

TL;DR

This paper introduces a supervised dictionary learning method that maximizes the dependency between signals and labels using HSIC, which can be kernelized for improved performance and efficiency.

Contribution

It presents a novel kernelized supervised dictionary learning approach that enhances class dependency modeling and outperforms existing methods.

Findings

Outperforms existing dictionary learning methods on real-world data

The approach is fast and produces compact dictionaries

Kernelization improves dependency maximization

Abstract

In this paper, we propose supervised dictionary learning (SDL) by incorporating information on class labels into the learning of the dictionary. To this end, we propose to learn the dictionary in a space where the dependency between the signals and their corresponding labels is maximized. To maximize this dependency, the recently introduced Hilbert Schmidt independence criterion (HSIC) is used. One of the main advantages of this novel approach for SDL is that it can be easily kernelized by incorporating a kernel, particularly a data-derived kernel such as normalized compression distance, into the formulation. The learned dictionary is compact and the proposed approach is fast. We show that it outperforms other unsupervised and supervised dictionary learning approaches in the literature, using real-world data.