Generalization Bounds on Multi-Kernel Learning with Mixed Datasets

TL;DR

This paper derives new generalization bounds for multi-kernel learning on mixed datasets modeled by Markov chains, accounting for dependencies among samples, with bounds depending on the number of kernels and training samples.

Contribution

It introduces novel generalization bounds for multi-kernel learning with dependent data from Markov chains, extending prior i.i.d. bounds to more realistic scenarios.

Findings

Bounds depend on $O(\sqrt{\log m})$ for kernels and $O(1/\sqrt{n})$ for samples.

Bounds account for sample dependencies, adding $O(1/\sqrt{n})$ terms.

Provides theoretical guarantees for learning in sensor networks and spatial-temporal models.

Abstract

This paper presents novel generalization bounds for the multi-kernel learning problem. Motivated by applications in sensor networks and spatial-temporal models, we assume that the dataset is mixed where each sample is taken from a finite pool of Markov chains. Our bounds for learning kernels admit $O(\sqrt{\log m})$ dependency on the number of base kernels and $O(1/\sqrt{n})$ dependency on the number of training samples. However, some $O(1/\sqrt{n})$ terms are added to compensate for the dependency among samples compared with existing generalization bounds for multi-kernel learning with i.i.d. datasets.