Learning from Distributions via Support Measure Machines

TL;DR

This paper introduces support measure machines (SMMs), a kernel-based framework that learns from probability distributions represented as mean embeddings in RKHS, extending traditional SVMs to operate on distributions rather than vectors.

Contribution

The paper develops support measure machines (SMMs), a novel extension of SVMs that directly learn from probability distributions, along with a flexible SVM variant (Flex-SVM) that uses different kernels per example.

Findings

SMMs effectively classify data represented as distributions.

Flex-SVM improves performance by customizing kernels for each training example.

Experimental results show the framework's success on synthetic and real-world data.

Abstract

This paper presents a kernel-based discriminative learning framework on probability measures. Rather than relying on large collections of vectorial training examples, our framework learns using a collection of probability distributions that have been constructed to meaningfully represent training data. By representing these probability distributions as mean embeddings in the reproducing kernel Hilbert space (RKHS), we are able to apply many standard kernel-based learning techniques in straightforward fashion. To accomplish this, we construct a generalization of the support vector machine (SVM) called a support measure machine (SMM). Our analyses of SMMs provides several insights into their relationship to traditional SVMs. Based on such insights, we propose a flexible SVM (Flex-SVM) that places different kernel functions on each training example. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework.