Communication-Efficient Distributed Online Learning with Kernels

TL;DR

This paper introduces a communication-efficient distributed online learning protocol for kernelized models, enabling real-time applications by reducing communication costs while maintaining predictive performance.

Contribution

It extends existing protocols to kernelized learners with model compression, providing a new criterion to bound communication by loss, improving efficiency in distributed online learning.

Findings

Supports larger class of online algorithms

Reduces communication costs in distributed settings

Maintains predictive performance with model compression

Abstract

We propose an efficient distributed online learning protocol for low-latency real-time services. It extends a previously presented protocol to kernelized online learners that represent their models by a support vector expansion. While such learners often achieve higher predictive performance than their linear counterparts, communicating the support vector expansions becomes inefficient for large numbers of support vectors. The proposed extension allows for a larger class of online learning algorithms---including those alleviating the problem above through model compression. In addition, we characterize the quality of the proposed protocol by introducing a novel criterion that requires the communication to be bounded by the loss suffered.