A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints

TL;DR

This paper introduces a novel decentralized optimization algorithm for problems with generalized orthogonality constraints, overcoming separability issues via a gradient and Jacobian tracking approach, with proven convergence and demonstrated effectiveness.

Contribution

The paper proposes a new algorithm that handles decentralized generalized orthogonality constraints by simultaneously tracking gradients and Jacobians, with theoretical convergence guarantees.

Findings

Algorithm converges globally with proven iteration complexity.

Effective on synthetic and real-world datasets.

Addresses separability issues in decentralized optimization.

Abstract

In this paper, we consider the decentralized optimization problems with generalized orthogonality constraints, where both the objective function and the constraint exhibit a distributed structure. Such optimization problems, albeit ubiquitous in practical applications, remain unsolvable by existing algorithms in the presence of distributed constraints. To address this issue, we convert the original problem into an unconstrained penalty model by resorting to the recently proposed constraint-dissolving operator. However, this transformation compromises the essential property of separability in the resulting penalty function, rendering it impossible to employ existing algorithms to solve. We overcome this difficulty by introducing a novel algorithm that tracks the gradient of the objective function and the Jacobian of the constraint mapping simultaneously. The global convergence guarantee is rigorously established with an iteration complexity. To substantiate the effectiveness and efficiency of our proposed algorithm, we present numerical results on both synthetic and real-world datasets.