Decentralized Dynamic Discriminative Dictionary Learning

TL;DR

This paper introduces a distributed online method for discriminative dictionary learning in networks, using a saddle point algorithm that ensures convergence to a stationary point despite non-convexity.

Contribution

It proposes a novel block Arrow-Hurwicz saddle point algorithm for distributed non-convex dictionary learning with theoretical convergence guarantees.

Findings

Algorithm converges to a first-order stationary point.

Only neighboring nodes exchange model information.

Effective for online, distributed discriminative learning.

Abstract

We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn a common set of dictionary elements of a feature space and model parameters while sequentially receiving observations. We formulate this problem as a distributed stochastic program with a non-convex objective and present a block variant of the Arrow-Hurwicz saddle point algorithm to solve it. Using Lagrange multipliers to penalize the discrepancy between them, only neighboring nodes exchange model information. We show that decisions made with this saddle point algorithm asymptotically achieve a first-order stationarity condition on average.