Distributed Online Learning with Multiple Kernels

TL;DR

This paper introduces DOMKL, a distributed online learning framework for IoT data that preserves privacy, achieves near-optimal regret, and ensures consensus among learners, validated through real-world experiments.

Contribution

The paper proposes a novel distributed online learning method combining OADMM and Hedge algorithms for IoT systems, with theoretical guarantees and experimental validation.

Findings

Achieves sublinear regret over T time slots.

Ensures consensus among neighboring learners.

Effective in regression and time-series prediction tasks.

Abstract

In the Internet-of-Things (IoT) systems, there are plenty of informative data provided by a massive number of IoT devices (e.g., sensors). Learning a function from such data is of great interest in machine learning tasks for IoT systems. Focusing on streaming (or sequential) data, we present a privacy-preserving distributed online learning framework with multiplekernels (named DOMKL). The proposed DOMKL is devised by leveraging the principles of an online alternating direction of multipliers (OADMM) and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret, implying that every learned function achieves the performance of the best function in hindsight as in the state-of-the-art centralized online learning method. Moreover, it is ensured that the learned functions of any two neighboring learners have a negligible difference as T grows, i.e., the so-called consensus constraints hold. Via experimental tests with various real datasets, we verify the effectiveness of the proposed DOMKL on regression and time-series prediction tasks.