On the Asymptotic Bias of the Diffusion-Based Distributed Pareto Optimization

TL;DR

This paper analyzes the asymptotic bias of diffusion-based distributed Pareto optimization algorithms, showing it diminishes linearly with small step-sizes and providing new insights into their behavior under practical conditions.

Contribution

It introduces an alternative, less complex analysis method for the asymptotic bias of diffusion Pareto optimization, especially when certain conditions are not strictly met.

Findings

Asymptotic bias decreases linearly with the largest step-size.

Provided an explicit expression for bias without strict condition assumptions.

Method offers new insights with reduced computational complexity.

Abstract

We revisit the asymptotic bias analysis of the distributed Pareto optimization algorithm developed based on the diffusion strategies. We propose an alternative way to analyze the asymptotic bias of this algorithm at small step-sizes and show that the asymptotic bias descends to zero with a linear dependence on the largest step-size parameter when this parameter is sufficiently small. In addition, through the proposed analytic approach, we provide an expression for the small-step-size asymptotic bias when a condition assumed jointly on the combination matrices and the step-sizes does not strictly hold. This is a likely scenario in practice, which has not been considered in the original paper that introduced the algorithm. Our methodology provides new insights into the inner workings of the diffusion Pareto optimization algorithm while being considerably less involved than the small-step-size asymptotic bias analysis presented in the original work. This is because we take advantage of the special eigenstructure of the composite combination matrix used in the algorithm without calling for any eigenspace decomposition or matrix inversion.