QC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM

TL;DR

This paper introduces QC-ODKLA, a communication-efficient online decentralized kernel learning algorithm that employs quantization and censoring strategies, achieving optimal regret and effective learning in streaming data scenarios.

Contribution

It proposes a novel framework combining random features, linearized ADMM, and communication strategies for efficient decentralized online kernel learning.

Findings

Achieves optimal sublinear regret of O(√T)

Reduces communication costs via quantization and censoring

Demonstrates effective learning and efficiency in numerical experiments

Abstract

This paper focuses on online kernel learning over a decentralized network. Each agent in the network receives continuous streaming data locally and works collaboratively to learn a nonlinear prediction function that is globally optimal in the reproducing kernel Hilbert space with respect to the total instantaneous costs of all agents. In order to circumvent the curse of dimensionality issue in traditional online kernel learning, we utilize random feature (RF) mapping to convert the non-parametric kernel learning problem into a fixed-length parametric one in the RF space. We then propose a novel learning framework named Online Decentralized Kernel learning via Linearized ADMM (ODKLA) to efficiently solve the online decentralized kernel learning problem. To further improve the communication efficiency, we add the quantization and censoring strategies in the communication stage and develop the Quantized and Communication-censored ODKLA (QC-ODKLA) algorithm. We theoretically prove that both ODKLA and QC-ODKLA can achieve the optimal sublinear regret $\mathcal{O}(\sqrt{T})$ over $T$ time slots. Through numerical experiments, we evaluate the learning effectiveness, communication, and computation efficiencies of the proposed methods.