Generative and Latent Mean Map Kernels

TL;DR

This paper introduces two new kernels, GMMK and LMMK, that extend mean map embeddings to probabilistic and latent variable models, improving similarity measures and generalization in sequence data analysis.

Contribution

The paper presents novel generative and latent mean map kernels that enhance the embedding of probability measures and latent models, with demonstrated benefits in sequence classification tasks.

Findings

GMMK provides smooth similarity measures with better regularization.

GMMK shows competitive or superior generalization error in experiments.

LMMK effectively incorporates latent variables into Hilbert space embeddings.

Abstract

We introduce two kernels that extend the mean map, which embeds probability measures in Hilbert spaces. The generative mean map kernel (GMMK) is a smooth similarity measure between probabilistic models. The latent mean map kernel (LMMK) generalizes the non-iid formulation of Hilbert space embeddings of empirical distributions in order to incorporate latent variable models. When comparing certain classes of distributions, the GMMK exhibits beneficial regularization and generalization properties not shown for previous generative kernels. We present experiments comparing support vector machine performance using the GMMK and LMMK between hidden Markov models to the performance of other methods on discrete and continuous observation sequence data. The results suggest that, in many cases, the GMMK has generalization error competitive with or better than other methods.