# Adaptation and learning over networks under subspace constraints -- Part I: Stability Analysis

## TL;DR

This paper introduces a distributed adaptive algorithm for network optimization under subspace constraints, proving its stability and small estimation errors in the small step-size regime, with further performance analysis in the sequel.

## Contribution

It develops a distributed implementation of projected gradient descent for subspace-constrained network optimization, ensuring stability and performance close to centralized solutions.

## Key findings

- Distributed algorithm achieves small estimation errors for small step-sizes.
- The proposed method generalizes consensus optimization to broader subspace constraints.
- Part II will analyze steady-state performance considering noise and data characteristics.

## Abstract

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional subspaces. This constrained formulation includes consensus optimization as a special case, and allows for more general task relatedness models such as smoothness. While such formulations can be solved via projected gradient descent, the resulting algorithm is not distributed. Starting from the centralized solution, we propose an iterative and distributed implementation of the projection step, which runs in parallel with the stochastic gradient descent update. We establish in this Part I of the work that, for small step-sizes $\mu$, the proposed distributed adaptive strategy leads to small estimation errors on the order of $\mu$. We examine in the accompanying Part II [2] the steady-state performance. The results will reveal explicitly the influence of the gradient noise, data characteristics, and subspace constraints, on the network performance. The results will also show that in the small step-size regime, the iterates generated by the distributed algorithm achieve the centralized steady-state performance.