# Adaptive Kernel Learning in Heterogeneous Networks

## TL;DR

This paper introduces HALK, a decentralized algorithm for heterogeneous networks that adaptively learns kernel-based regression functions while satisfying nonlinear proximity constraints, achieving improved convergence rates.

## Contribution

It proposes a novel decentralized stochastic primal-dual method with adaptive kernel projections for heterogeneous networks, enhancing convergence and constraint satisfaction.

## Key findings

- Achieves (\u221a{T}) sub-optimality attenuation with constant step-sizes.
- Ensures long-term satisfaction of nonlinear proximity constraints.
- Improves upon existing rates for vector-valued decentralized learning.

## Abstract

We consider learning in decentralized heterogeneous networks: agents seek to minimize a convex functional that aggregates data across the network, while only having access to their local data streams. We focus on the case where agents seek to estimate a regression \emph{function} that belongs to a reproducing kernel Hilbert space (RKHS). To incentivize coordination while respecting network heterogeneity, we impose nonlinear proximity constraints. To solve the constrained stochastic program, we propose applying a functional variant of stochastic primal-dual (Arrow-Hurwicz) method which yields a decentralized algorithm. To handle the fact that agents' functions have complexity proportional to time (owing to the RKHS parameterization), we project the primal iterates onto subspaces greedily constructed from kernel evaluations of agents' local observations. The resulting scheme, dubbed Heterogeneous Adaptive Learning with Kernels (HALK), when used with constant step-sizes, yields $\mathcal{O}(\sqrt{T})$ attenuation in sub-optimality and exactly satisfies the constraints in the long run, which improves upon the state of the art rates for vector-valued problems.